Unity + Deep Learning Support

Discussion in 'Feature Requests' started by Amin, Aug 21, 2017.

  1. Hi

    I know that in the reference page it says that we can use MuJoCo along with advanced rendering engines such as Unity; but I believe for that we can only work with C/C++ interface. Is that right?
    I'm an AI researcher working on deep learning and most of the top deep learning libraries are currently written in Python. So a very important feature for me is to be able to make MuJoCo-Py library to work with Unity. Is that possible?

  2. Emo Todorov

    Emo Todorov Administrator Staff Member

    Probably, but you would have to do the work yourself. There are two approaches:

    1. Get MuJoCo and Unity to work together in C++ and wrap the whole thing in Python;
    2. Wrap both MuJoCo and Unity in Python separately, and then write Python code to exchange data between them as needed.

    I don't have experience with Unity or Unreal Engine, but my understanding is that these tools expect the developer to adopt their framework (that is why they are game engines and not just rendering engines), so integrating them with Python and using them as light-weight rendering engines for deep learning may be tricky...