search

emscripten-core / emsdk

Emscripten SDK
http://emscripten.org
Other
3.03k stars 693 forks source link

Already installed tools are not getting recognized as such #299

Open Morilli opened 5 years ago

Morilli commented 5 years ago

Running a .\emsdk install latest re-installs all tools except for the fastcomp release, even if they already exist. I believe the same check that detects that fastcomp already exists and has the correct contents could be applied to the other tools as well.

kripken commented 5 years ago

Interesting, it appears to work properly for me on linux:

$ ./emsdk install latest
Installing SDK 'sdk-releases-fastcomp-80bff2784f8500c1305ca69ba1d9fc84df0e401c-64bit'..
Installing tool 'releases-fastcomp-80bff2784f8500c1305ca69ba1d9fc84df0e401c-64bit'..
The contents of file 'https://storage.googleapis.com/webassembly/emscripten-releases-builds/linux/80bff2784f8500c1305ca69ba1d9fc84df0e401c/wasm-binaries.tbz2' already exist in destination '/home/azakai/Dev/sdk/fastcomp', skipping.
Done installing tool 'releases-fastcomp-80bff2784f8500c1305ca69ba1d9fc84df0e401c-64bit'.
Installing tool 'node-8.9.1-64bit'..
The contents of file 'node-v8.9.1-linux-x64.tar.xz' already exist in destination '/home/azakai/Dev/sdk/node/8.9.1_64bit', skipping.
Done installing tool 'node-8.9.1-64bit'.
Done installing SDK 'sdk-releases-fastcomp-80bff2784f8500c1305ca69ba1d9fc84df0e401c-64bit'.
$ ./emsdk install latest
Installing SDK 'sdk-releases-fastcomp-80bff2784f8500c1305ca69ba1d9fc84df0e401c-64bit'..
Installing tool 'releases-fastcomp-80bff2784f8500c1305ca69ba1d9fc84df0e401c-64bit'..
The contents of file 'https://storage.googleapis.com/webassembly/emscripten-releases-builds/linux/80bff2784f8500c1305ca69ba1d9fc84df0e401c/wasm-binaries.tbz2' already exist in destination '/home/azakai/Dev/sdk/fastcomp', skipping.
Done installing tool 'releases-fastcomp-80bff2784f8500c1305ca69ba1d9fc84df0e401c-64bit'.
Installing tool 'node-8.9.1-64bit'..
The contents of file 'node-v8.9.1-linux-x64.tar.xz' already exist in destination '/home/azakai/Dev/sdk/node/8.9.1_64bit', skipping.
Done installing tool 'node-8.9.1-64bit'.
Done installing SDK 'sdk-releases-fastcomp-80bff2784f8500c1305ca69ba1d9fc84df0e401c-64bit'.

Perhaps the problem is windows-specific? Or am I misreading the logging?

Morilli commented 5 years ago

I just tested on my WSL and indeed, running python3 ./emsdk install latest there exits immediately, telling me all tools are already installed. On Windows, it's always re-downloading the zips and re-extracting them, instead of recognizing that their contents already exist.

Morilli commented 5 years ago

Actually, looking at it some more, it seems that my linux only tries to install nodejs in addition to the fastcomp, whereas windows also installs java and python. Zips for those two are not saved, therefore they aren't recognized as "existent".

kripken commented 5 years ago

Do you get the problem only on java and python, and not on nodejs?

If you do, then it might be a windows-only bug. If you don't, then maybe it's some difference between java/python and between nodejs. (but looking in the manifest, I don't see an obvious difference)

If it's the former, then maybe it's that we use zip files on windows but not elsewhere for those downloads. Looks like cleanup_temp_install_files is not aware of all suffixes yet, might be related.

Morilli commented 5 years ago

On Linux I'm not getting any issues with the download whatsoever, but on windows the three tools nodejs, java and python do not save their zips, which means the check for these zips will fail and they will re-download. So the main issue is gonna be the fact that these zip files are deleted after they are extracted. Not sure whether I feel like these zip files should logically even be kept (regarding the fact the fastcomp one takes up solid 700 MB on windows), but that's probably a different issue.

SagaraBattousai commented 5 years ago

I'm only getting an issue with fastcomp, on windows (through powershell and cmd with python 3) it says: python .\emsdk install latest Warning: unknown machine architecture AMD64

Error: No tool or SDK found by name 'sdk-releases-fastcomp-7b4b328af02eafbc857b8ca1e3d9b12dddc56ef7-64bit'

but on WSL : python3 ./emsdk install latest Installing SDK 'sdk-releases-fastcomp-7b4b328af02eafbc857b8ca1e3d9b12dddc56ef7-64bit'.. Installing tool 'releases-fastcomp-7b4b328af02eafbc857b8ca1e3d9b12dddc56ef7-64bit'.. Downloading: /mnt/c/Users/james/emsdk/zips/7b4b328af02eafbc857b8ca1e3d9b12dddc56ef7-wasm-binaries.tbz2 from https://storage.googleapis.com/webassembly/emscripten-releases-builds/linux/7b4b328af02eafbc857b8ca1e3d9b12dddc56ef7/wasm-binaries.tbz2, 159109971 Bytes 76013568 [47.77%]

and appears to be working, would this mean that it's the windows version of python 3 thats causing the issue?

xu6767 commented 5 years ago

I also have that ‘Warning: unknown machine architecture AMD64’ warning，when runding ‘emsdk install latest ’ on windows ，how do you fix it ？

SagaraBattousai commented 5 years ago

I also have that ‘Warning: unknown machine architecture AMD64’ warning，when runding ‘emsdk install latest ’ on windows ，how do you fix it ？

It should be fixed on master branch now, you need to add .lower() to the emsdk script (line 85) or just git pull.