Linker error: unresolved external symbol

Discussion in 'MuJoCo HAPTIX Bug Reports' started by wlorenz65, Jun 21, 2016.

  1. When trying to build grasp.exe from a Visual Studio 2012 command prompt, the linker complains about 7 unresolved externals:
    Code:
    H:\Desktop\mjhaptix131\apicpp>nmake
    
    Microsoft (R) Program Maintenance Utility Version 12.00.21005.1
    Copyright (C) Microsoft Corporation.  All rights reserved.
    
            cl /O2 /MT /EHsc grasp.cpp mjhaptix_user.lib
    Microsoft (R) C/C++ Optimizing Compiler Version 18.00.31101 for x86
    Copyright (C) Microsoft Corporation.  All rights reserved.
    
    grasp.cpp
    Microsoft (R) Incremental Linker Version 12.00.31101.0
    Copyright (C) Microsoft Corporation.  All rights reserved.
    
    /out:grasp.exe
    grasp.obj
    mjhaptix_user.lib
    grasp.obj : error LNK2019: unresolved external symbol __imp__hx_connect referenced in function _main
    grasp.obj : error LNK2019: unresolved external symbol __imp__hx_close referenced in function _main
    grasp.obj : error LNK2019: unresolved external symbol __imp__hx_robot_info referenced in function _main
    grasp.obj : error LNK2019: unresolved external symbol __imp__hx_update referenced in function _main
    grasp.obj : error LNK2019: unresolved external symbol __imp__hx_read_sensors referenced in function _main
    grasp.obj : error LNK2019: unresolved external symbol __imp__hx_double_time referenced in function _main
    grasp.obj : error LNK2019: unresolved external symbol __imp__mj_message referenced in function _main
    grasp.exe : fatal error LNK1120: 7 unresolved externals
    NMAKE : fatal error U1077: '"C:\Program Files (x86)\Microsoft Visual Studio 12.0\VC\BIN\cl.EXE"' : return code '0x2'
    Stop.
    
    H:\Desktop\mjhaptix131\apicpp>
    Examining the exported symbols from the dll and the lib, dumpbin says:
    Code:
    H:\Desktop\mjhaptix131\apicpp>dumpbin mjhaptix_user.dll /exports
    Microsoft (R) COFF/PE Dumper Version 12.00.31101.0
    Copyright (C) Microsoft Corporation.  All rights reserved.
    
    
    Dump of file mjhaptix_user.dll
    
    File Type: DLL
    
      Section contains the following exports for mjhaptix_user.dll
    
        00000000 characteristics
        571C414C time date stamp Sun Apr 24 05:45:16 2016
            0.00 version
               1 ordinal base
              41 number of functions
              41 number of names
    
        ordinal hint RVA      name
    
              1    0 00001600 hx_close
              2    1 00001550 hx_connect
              3    2 00001F80 hx_double_time
              4    3 00001ED0 hx_last_result
              5    4 00001E70 hx_read_sensors
              6    5 00001680 hx_robot_info
              7    6 00001840 hx_update
              8    7 000039C0 mj_close
              9    8 000039A0 mj_connect
             10    9 00003A50 mj_connected
             11    A 00004120 mj_equality
             12    B 00002E50 mj_get_actuator
             13    C 00002260 mj_get_applied
             14    D 00002930 mj_get_body
             15    E 00003130 mj_get_contact
             16    F 00002130 mj_get_control
             17   10 000026A0 mj_get_dynamics
             18   11 00002FD0 mj_get_force
             19   12 00002A70 mj_get_geom
             20   13 00002560 mj_get_mocap
             21   14 00002410 mj_get_onebody
             22   15 000037C0 mj_get_rgba
             23   16 00002800 mj_get_sensor
             24   17 00002BB0 mj_get_site
             25   18 00001FA0 mj_get_state
             26   19 00002CF0 mj_get_tendon
             27   1A 000043B0 mj_handler
             28   1B 00004300 mj_id2name
             29   1C 00003A60 mj_info
             30   1D 00004180 mj_message
             31   1E 00004250 mj_name2id
             32   1F 000040B0 mj_reset
             33   20 00003A40 mj_result
             34   21 000034E0 mj_set_applied
             35   22 00003430 mj_set_control
             36   23 000036F0 mj_set_mocap
             37   24 00003630 mj_set_onebody
             38   25 000038B0 mj_set_rgba
             39   26 00003300 mj_set_state
             40   27 00003EC0 mj_step
             41   28 00003F10 mj_update
    
      Summary
    
           1B000 .data
            1000 .gfids
            2000 .pdata
            B000 .rdata
            1000 .reloc
            1000 .rsrc
           16000 .text
    
    H:\Desktop\mjhaptix131\apicpp>dumpbin mjhaptix_user.lib /exports
    Microsoft (R) COFF/PE Dumper Version 12.00.31101.0
    Copyright (C) Microsoft Corporation.  All rights reserved.
    
    
    Dump of file mjhaptix_user.lib
    
    File Type: LIBRARY
    
         Exports
    
           ordinal    name
    
                      hx_close
                      hx_connect
                      hx_double_time
                      hx_last_result
                      hx_read_sensors
                      hx_robot_info
                      hx_update
                      mj_close
                      mj_connect
                      mj_connected
                      mj_equality
                      mj_get_actuator
                      mj_get_applied
                      mj_get_body
                      mj_get_contact
                      mj_get_control
                      mj_get_dynamics
                      mj_get_force
                      mj_get_geom
                      mj_get_mocap
                      mj_get_onebody
                      mj_get_rgba
                      mj_get_sensor
                      mj_get_site
                      mj_get_state
                      mj_get_tendon
                      mj_handler
                      mj_id2name
                      mj_info
                      mj_message
                      mj_name2id
                      mj_reset
                      mj_result
                      mj_set_applied
                      mj_set_control
                      mj_set_mocap
                      mj_set_onebody
                      mj_set_rgba
                      mj_set_state
                      mj_step
                      mj_update
    
      Summary
    
              D5 .debug$S
              14 .idata$2
              14 .idata$3
               8 .idata$4
               8 .idata$5
              12 .idata$6
    
    H:\Desktop\mjhaptix131\apicpp>
    Because the ordinals are missing from the lib, the linker has to use the names. But those exported names have no leading underscore!

    Here are the names the linker would like to see:
    Code:
    with __declspec(dllimport) and extern "C":
    
        /Gd __cdecl (default)    __imp__hx_connect
        /Gr __fastcall           __imp_@hx_connect@8
        /Gz __stdcall            __imp__hx_connect@8
        /Gv __vectorcall          __imp_hx_connect@@8
    
    with #define MJ_STATIC and extern "C":
    
        /Gd __cdecl (default)          _hx_connect
        /Gr __fastcall                 @hx_connect@8
        /Gz __stdcall                  _hx_connect@8
        /Gv __vectorcall                hx_connect@@8
    
    with __declspec(dllimport) and without extern "C":
    
        /Gd __cdecl (default)    __imp_?hx_connect@@YA?AW4hxResult@@PBDH@Z
        /Gr __fastcall           __imp_?hx_connect@@YI?AW4hxResult@@PBDH@Z
        /Gz __stdcall            __imp_?hx_connect@@YG?AW4hxResult@@PBDH@Z
        /Gv __vectorcall         __imp_?hx_connect@@YQ?AW4hxResult@@PBDH@Z
    
    with #define MJ_STATIC and without extern "C":
    
        /Gd __cdecl (default)          ?hx_connect@@YA?AW4hxResult@@PBDH@Z
        /Gr __fastcall                 ?hx_connect@@YI?AW4hxResult@@PBDH@Z
        /Gz __stdcall                  ?hx_connect@@YG?AW4hxResult@@PBDH@Z
        /Gv __vectorcall               ?hx_connect@@YQ?AW4hxResult@@PBDH@Z
    The static linkage with extern "C" comes closest to the names that the library exports, but it has an additional leading underscore.

    Is the mjhaptix131 apicpp not compatible with Visual Studo 2012? At the moment, I don't want to upgrade to a newer version or use another compiler.
     
  2. Emo Todorov

    Emo Todorov Administrator Staff Member

    MuJoCo HAPTIX is compiled with Visual Studio 2015, and nmake works fine with the grasp.cpp example. When I tried to compile and link without the mjhaptix_user.lib file, I got the following:

    Code:
    C:\released\mjhaptix131\apicpp>cl /O2 /MT /EHsc grasp.cpp
    Microsoft (R) C/C++ Optimizing Compiler Version 19.00.23918 for x64
    Copyright (C) Microsoft Corporation.  All rights reserved.
    
    grasp.cpp
    Microsoft (R) Incremental Linker Version 14.00.23918.0
    Copyright (C) Microsoft Corporation.  All rights reserved.
    
    /out:grasp.exe
    grasp.obj
    grasp.obj : error LNK2019: unresolved external symbol __imp_hx_connect referenced in function main
    grasp.obj : error LNK2019: unresolved external symbol __imp_hx_close referenced in function main
    grasp.obj : error LNK2019: unresolved external symbol __imp_hx_robot_info referenced in function main
    grasp.obj : error LNK2019: unresolved external symbol __imp_hx_update referenced in function main
    grasp.obj : error LNK2019: unresolved external symbol __imp_hx_read_sensors referenced in function main
    grasp.obj : error LNK2019: unresolved external symbol __imp_hx_double_time referenced in function main
    grasp.obj : error LNK2019: unresolved external symbol __imp_mj_message referenced in function main
    grasp.exe : fatal error LNK1120: 7 unresolved externals
    Comparing to the link errors you get, this is almost identical except you have an extra underscore before the function names. So one possibility is that Microsoft changed the number of automatically-generated underscores between compiler/linker versions... this seems unlikely though.

    As for the combination of defines etc, do not define MJ_STATIC, this is for internal use only.

    I wonder if the following could be the problem. In haptix.h, we have:

    Code:
    // this is a C API
    #if defined(__cplusplus)
    extern "C" {
    #endif
    Could it be that "__cplusplus" is not automatically defined in VS12? This is easy for you to test: edit haptix.h to remove the conditional and always use extern "C".
     
  3. Emo Todorov

    Emo Todorov Administrator Staff Member

    Here is another idea. grasp.cpp happens to be a legitimate C file. So you can change the file extension from .cpp to .c and try again. With VS15 this worked for me, but then again, the original also worked...
     
  4. Issue solved: mjhaptix_user.lib is an x64 library, but the Visual Studio 2012 command line tools default to x86. Seems that Microsoft changed this in VS2015. With VS2012, I had to call vcvars.bat x86 first before running nmake. Now grasp.exe compiles and runs out of the box.
     
  5. Emo Todorov

    Emo Todorov Administrator Staff Member

    You would think that a 32-64 bit mismatch would trigger a more informative message than "unresolved external symbol" :)
     
  6. You don't know Intel C++ 9.1... This was a replacement for Visual C++ 2005 which created somewhat faster executables. Its error messages were so confusing and unclear, that I had to give up using it for development.

    The messages from Visual C++'s compiler are normally detailed and informative. But this 32bit-64bit mismatch is a linker issue. It's the same with the loader in Windows 7. I could get a 32bit grasp.exe linked with the 64bit mjhaptix_user.lib by creating a DEF file with aliases. The resulting executable loaded mjhaptix_user.dll and then aborted with error 0xC00000B7 or something similar. Error lookup said that there is no explanatory message for this number.

    Maybe there are different teams at Microsoft for the compiler and the linker/loader. Maybe the linker/loader team is for old developers who have been put in a backwater. They don't care about user friendlyness. Maybe in 2012/2013, 64bit executables were too rare. Maybe Microsoft expects everybody to upgrade to Windows 10 and Visual Studio 2015. I don't know.
     
  7. Emo Todorov

    Emo Todorov Administrator Staff Member

    In pursuit of user-friendliness, the Windows 10 update is automatic whether you want it or not! Still, having just wasted two days with Ubuntu 16.04 installations that get stuck in infinite login loops or broken wifi, as well as OSX drivers that "resolve" multi-sample depth buffers by giving you only the left half of the video frame, I would rather stick to Windows/VS for development. Unlike OSX where things either work or they don't and there is nothing you can do, there is usually a way to make anything work on Windows. And unlike Linux where you have to Google every step of the way, only to discover 10 different ways of doing the same thing, most of which don't work in your situation, Windows tends to be more self-explanatory. Still a mess, but the lesser evil in my book.

    Btw, have you compared the Intel compiler to VS or GCC recently? Last I tested was about a year ago, and the MuJoCo timing tests did not show consistent advantages of either compiler. A few years earlier Intel was definitely producing faster numerical code, but now VS and GCC may have caught up. Maybe I should test again...
     
  8. I've spent the last few years reading scientific papers about A.I. and robotics. In 2009, I took my game-benchmarks-with-vision suite offline and stayed away from compilers since then. So no, I didn't check Intel's compiler recently.

    I'm wondering why, on the one hand, runtime performance seems so important for you, but on the other hand, MuJoCo uses the same timestep for all objects. If two rigid objects with hard contacts collide, the timestep cannot be small enough at that moment, or else the simulation will become physically implausible or instable. On the other hand, if there are 50 contacts silently sitting in a corner with constant contact forces, they don't need a small timestep. Forcing all contacts to use the same timestep is a somewhat wasteful concept, performancewise.

    Well, I'm no expert in physical simulation. I wrote a C64 simulator between 1992 and 1997, but the C64 is man-made. These different timesteps may be too complicated to implement, or the performance gain is too small and not worth the effort.
     
  9. Emo Todorov

    Emo Todorov Administrator Staff Member

    W have to assume that the system is coupled, so the entire simulation has to be integrated at the same time step. You could have disconnected groups of objects occasionally, but the group membership will change dynamically and has to be discovered online, and solving the underlying graph partitioning problem would cancel any savings. It is possible to adapt the time step over time, using RK45 for example. Currently we have the fixed-step RK4, and it seems to be beneficial only in the absence of contacts -- which makes me doubt that RK45 will add much. It is on the to-do list though.

    As for using very small steps when new contacts are formed, this was the case with spring-damper models, which is why such models have fallen out of favor. The modern approach to contact modelling allows large steps. This is true both for LCP-type models (used in most other simulators) and for MuJoCo's soft convex model.

    In general, whenever it looks like there are computational savings to be had, I go after them. But they have to be valid for any system that can be represented in MuJoCo's modeling format. This is very restrictive. As a result, many tricks that look like possible savings turn out not to produce valid simulations, or to be slower once implemented in a way that supports all possible models.