upgrade ml-agents version

[joanllobera]

Dear @Sohojoe , I see this version uses version 0.5 of ml-agents, when current version is 0.8. I have tested it with Unity 2019.1, and it seems to work, at least what I tested so far.

I have tried to do a brutal upgrade, changing all the files in your repo with the updated ones from the ml-agents and the marathon repos. However, your project has dependencies on both ml-agents and the marathon framework that are not compatible anymore. The latest versions several dependencies which were interfaces are now full classes, and they also have some changes in the APIs, so your code generates errors when used with the latest versions of ml-agents and marathon.

I am wondering if you have any plan to do update your repo to address these issues, I am interested in getting a functional version of the Deepmimics paper in this framework. Regards, Joan

[Sohojoe]

Hi @joanllobera, thanks for your note. I am supporting the project and have been helping others reproduce DeepMimic results with it.

I've heald back from updating the more current ml-agents as I was hoping to integrate better with MarathonEnvs - however, I got pulled into some other research projects so have not taken the time to do that properly.

I'm open to doing a quick update to just the repro if that is helpful. Something that would help me is understanding how you hope to use the project (for example, is there any specific feature of mlagents 0.8 that you need? How are you hoping to use this?)

Note: I'm keen to see how well the SAC (soft actor-critic) algorithm performs, it looks like Unity is implementing a version of SAC into ML-Agents, however, it's in the development branch

[joanllobera]

Hi, I want to explore whether we can use these active ragdolls to create different kind of behaviours.

My interest is not so much on complicated behaviours, like backflips, but rather exploring time-related constraints, and different ways to parameterize different behaviours (a reaching gesture, for example)

I also plan to add this into a machine learning course for videogame developers that I gave ast spring and will give again next spring.

For this last reason, I need it to be compatible with the rest of the materials. In mlagents from 0.5 to 0.6 they did some architecture and api changes that would need some adjustments in your code. And 0.8 continues with the same architecture than 0.6, as far as I can tell.

I also think the more recent marathon versions have integrated with these recent versions of mlagents, and changed the api. when I tried the update some of your classes were giving errors related to inheritance from marathon classes.

It should not be too much of a deal, but if I do it myself it will require some time to understand your class structure. So if it is possible to have a bit of help on that it would be very much apreciated.

I will also look a bit into the SAC that you point to, thanks for pointing that out. Regards, Joan

On Sat, 13 Jul 2019, 22:35 Joe Booth, notifications@github.com wrote:

Hi @joanllobera https://github.com/joanllobera, thanks for your note. I am supporting the project and have been helping others reproduce DeepMimic results with it.

I've heald back from updating the more current ml-agents as I was hoping to integrate better with MarathonEnvs - however, I got pulled into some other research projects so have not taken the time to do that properly.

I'm open to doing a quick update to just the repro if that is helpful. Something that would help me is understanding how you hope to use the project (for example, is there any specific feature of mlagents 0.8 that you need? How are you hoping to use this?)

Note: I'm keen to see how well the SAC (soft actor-critic) algorithm performs, it looks like Unity is implementing a version of SAC into ML-Agents, however, it's in the development branch

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/Sohojoe/ActiveRagdollStyleTransfer/issues/3?email_source=notifications&email_token=AA5H4TCCYBX6K6XSEW24OPTP7I4B7A5CNFSM4ICKPCM2YY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGODZ3Y6AQ#issuecomment-511151874, or mute the thread https://github.com/notifications/unsubscribe-auth/AA5H4TAAXCJEJCVJ7S4LDL3P7I4B7ANCNFSM4ICKPCMQ .

[Sohojoe]

Hi Joan, I should be able to take a look later this week - will keep you posted!

[Sohojoe]

@joanllobera I have a version running against the latest ML-agents as a branch in my marathon-envs - https://github.com/Sohojoe/marathon-envs/tree/feature/add_style_transfer

I retrained the backflip and will add some more mocaps - my goal is to have it as a set of environments for marathon-envs

[joanllobera]

that is great. At the moment I am busy preparing a paper submission, but as soon as i am done i will look into this in quite some detail. Many thanks! Joan

Missatge de Joe Booth notifications@github.com del dia dv., 30 d’ag. 2019 a les 0:15:

@joanllobera https://github.com/joanllobera I have a version running against the latest ML-agents as a branch in my marathon-envs - https://github.com/Sohojoe/marathon-envs/tree/feature/add_style_transfer

I retrained the backflip and will add some more mocaps - my goal is to have it as a set of environments for marathon-envs

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/Sohojoe/ActiveRagdollStyleTransfer/issues/3?email_source=notifications&email_token=AA5H4TASUMO436HGSQQ373DQHBDATA5CNFSM4ICKPCM2YY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOD5P7MOA#issuecomment-526382648, or mute the thread https://github.com/notifications/unsubscribe-auth/AA5H4TB6HYWLJH6CBZMN34LQHBDATANCNFSM4ICKPCMQ .

--

Joan Llobera - http://joan.llobera.name joan.llobera@gmail.com

[joanllobera]

hi Joe, I am working on the branch that you pointed to me.

I am having trouble reproducing your training procedure. Specifically, when I do:

conda create -n marathon-ml09 conda activate marathon-ml09 conda install python=3.6 pip install -e .

I get this message: ERROR: mlagents-envs 0.9.0 has requirement Pillow<=5.4.1,>=4.2.1, but you'll have pillow 6.1.0 which is incompatible.

The installation completes, but then when I do try to train, it does not work. So, I uninstall Pillow, reinstall that version. Doing this, it works in editor mode (although very slow), but not with a compiled project.

In the compiled version, the compiled environment is ofund, at opens. However, only a the skybox of the unity project appears, and ml-agents complains about a broken pipe, essentially that Unity takes too long to respond. The error looks like:

... subprocess_env_manager.py", line 58, in recv raise UnityCommunicationException("UnityEnvironment worker: recv failed.") mlagents.envs.exception.UnityCommunicationException: UnityEnvironment worker: recv failed.

I am wondering if I should change the main scene in any way? At the moment all I have done is turn on Training Mode, on object Academy, in the editor

Also, what is the most useful of this for others? should I document this on the readme, in the branch where you have implemetned Style Transfer, how to get this to work (as far as I got)?

Thanks, Joan

ERROR: mlagents-envs 0.9.0 has requirement Pillow<=5.4.1,>=4.2.1, but you'll have pillow 6.1.0 which is incompatible.

Missatge de joan joan.llobera@gmail.com del dia dv., 30 d’ag. 2019 a les 10:19:

that is great. At the moment I am busy preparing a paper submission, but as soon as i am done i will look into this in quite some detail. Many thanks! Joan

Missatge de Joe Booth notifications@github.com del dia dv., 30 d’ag. 2019 a les 0:15:

@joanllobera https://github.com/joanllobera I have a version running against the latest ML-agents as a branch in my marathon-envs - https://github.com/Sohojoe/marathon-envs/tree/feature/add_style_transfer

I retrained the backflip and will add some more mocaps - my goal is to have it as a set of environments for marathon-envs

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/Sohojoe/ActiveRagdollStyleTransfer/issues/3?email_source=notifications&email_token=AA5H4TASUMO436HGSQQ373DQHBDATA5CNFSM4ICKPCM2YY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOD5P7MOA#issuecomment-526382648, or mute the thread https://github.com/notifications/unsubscribe-auth/AA5H4TB6HYWLJH6CBZMN34LQHBDATANCNFSM4ICKPCMQ .

--

Joan Llobera - http://joan.llobera.name joan.llobera@gmail.com

--

Joan Llobera - http://joan.llobera.name joan.llobera@gmail.com

[Sohojoe]

@joanllobera - I will look at this over the weekend!

[joanllobera]

Hi, I have all tried installing with pip install mlagents, and I get essentially the same result.

I am happy t try other options. Meanwhile next week I will try with your develop branch and see how far I get

On Fri, 6 Sep 2019, 17:57 Joe Booth, notifications@github.com wrote:

@joanllobera https://github.com/joanllobera - I will look at this over the weekend!

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/Sohojoe/ActiveRagdollStyleTransfer/issues/3?email_source=notifications&email_token=AA5H4TGJFEUDPTD3NZ4BJKLQIJ4WDA5CNFSM4ICKPCM2YY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOD6DI4EY#issuecomment-528911891, or mute the thread https://github.com/notifications/unsubscribe-auth/AA5H4THTROOCEOTM75F7WC3QIJ4WDANCNFSM4ICKPCMQ .

[Sohojoe]

hi @joanllobera - I pushed a fix by making my modified ml-agents have the same Pillow requirement as ml-agents-envs. That was the underlying issue.

You don't need to make any changes to the unity environment and it should work for showcasing the pre-trained model as well as for training.

When trying - make sure you have a version of TensorFlow 1.7.x that supports AVX2 FMA (assuming your CPU supports them; most do)

[joanllobera]

hi Joe, it works! Now i am looking into how to have a tensorflow build that supports AVX2 thanks Joe, this is great.

Joan

Missatge de Joe Booth notifications@github.com del dia dt., 10 de set. 2019 a les 4:09:

hi @joanllobera https://github.com/joanllobera - I pushed a fix by making my modified ml-agents have the same Pillow requirement as ml-agents-envs. That was the underlying issue.

You don't need to make any changes to the unity environment and it should work for showcasing the pre-trained model as well as for training.

When trying - make sure you have a version of TensorFlow 1.7.x that supports AVX2 FMA (assuming your CPU supports them; most do)

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/Sohojoe/ActiveRagdollStyleTransfer/issues/3?email_source=notifications&email_token=AA5H4TGHG3XXVLAZ5C35OJTQI36WRA5CNFSM4ICKPCM2YY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOD6JSQCA#issuecomment-529737736, or mute the thread https://github.com/notifications/unsubscribe-auth/AA5H4TGAG5VOTIIDSN7POKLQI36WRANCNFSM4ICKPCMQ .

--

Joan Llobera - http://joan.llobera.name joan.llobera@gmail.com

[Sohojoe]

@joanllobera - here are links to AVX2 wheels for windows and Mac.

MacOS - https://github.com/yaroslavvb/tensorflow-community-wheels/issues/60 Windows CPU/GPU - https://github.com/fo40225/tensorflow-windows-wheel/tree/master/1.7.1/py36

[joanllobera]

dear Joe, just writing to let you know that I managed to start the training in Windows, with AVX2. There were some minor differences between triggering the training with and without AVX2. This was a bit puzzling, but otherwise it seems to work well.

Now I will start more systematic tests, first to check whether I can run it on linux, and then with other movements, replacing the backflip with something else. If this works, then I should be able to use it for my main purpose, which is related to the introduction of movements that respect certain temporal constraints.

cheers, Joan

Missatge de Joe Booth notifications@github.com del dia dv., 27 de set. 2019 a les 23:33:

@joanllobera https://github.com/joanllobera - here are links to AVX2 wheels for windows and Mac.

MacOS - yaroslavvb/tensorflow-community-wheels#60 https://github.com/yaroslavvb/tensorflow-community-wheels/issues/60 Windows CPU/GPU - https://github.com/fo40225/tensorflow-windows-wheel/tree/master/1.7.1/py36

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/Sohojoe/ActiveRagdollStyleTransfer/issues/3?email_source=notifications&email_token=AA5H4TE7E5F2HUQAJDTFUBTQLZ3Y5A5CNFSM4ICKPCM2YY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOD72EUVQ#issuecomment-536103510, or mute the thread https://github.com/notifications/unsubscribe-auth/AA5H4TBHEOVG2RHN6DG5TRDQLZ3Y5ANCNFSM4ICKPCMQ .

--

Joan Llobera - http://joan.llobera.name joan.llobera@gmail.com

[joanllobera]

dear Joe (@SohoJoe ) , I did not manage to get the character do the backflip, even with 3 days of training. It does imitate the movement, but it did not find the solution to do the backflip, it only "pretends" to do it, without falling. Next step is to try with different seeds, but at the moment compilation for linux crashes, for some obscure reason, and on windows it seems too slow, I guess I must be doing something wrong.

I am taking care of another experiment at the moment, so I might not be able to continue these tests until late january.

Meanwhile, I am wondering if it would be possible to contact you on a personal email. Out from my research efforts I am asked to deliver a proposal to implement an innovative VR experience, and I would like to know if you would be available to help us use this library for character animation in that particular project, as a freelance contractor. Your website joebooth.com does not seem to work.

my email is joan dot llobera at gmail dot com please send contact info (email and phone or skype).

Thanks! Regards, Joan

Missatge de joan joan.llobera@gmail.com del dia dt., 8 d’oct. 2019 a les 10:54:

dear Joe, just writing to let you know that I managed to start the training in Windows, with AVX2. There were some minor differences between triggering the training with and without AVX2. This was a bit puzzling, but otherwise it seems to work well.

Now I will start more systematic tests, first to check whether I can run it on linux, and then with other movements, replacing the backflip with something else. If this works, then I should be able to use it for my main purpose, which is related to the introduction of movements that respect certain temporal constraints.

cheers, Joan

Missatge de Joe Booth notifications@github.com del dia dv., 27 de set. 2019 a les 23:33:

@joanllobera https://github.com/joanllobera - here are links to AVX2 wheels for windows and Mac.

MacOS - yaroslavvb/tensorflow-community-wheels#60 https://github.com/yaroslavvb/tensorflow-community-wheels/issues/60 Windows CPU/GPU - https://github.com/fo40225/tensorflow-windows-wheel/tree/master/1.7.1/py36

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/Sohojoe/ActiveRagdollStyleTransfer/issues/3?email_source=notifications&email_token=AA5H4TE7E5F2HUQAJDTFUBTQLZ3Y5A5CNFSM4ICKPCM2YY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOD72EUVQ#issuecomment-536103510, or mute the thread https://github.com/notifications/unsubscribe-auth/AA5H4TBHEOVG2RHN6DG5TRDQLZ3Y5ANCNFSM4ICKPCMQ .

--

Joan Llobera - http://joan.llobera.name joan.llobera@gmail.com