sys not defined!

[philomorphism]

Downloaded this repo into test111 directory

V:\Yaan\learn\test111>where python C:\Users\XinYaanZyoy\Anaconda3\python.exe V:\Yaan\learn\test111>python --version Python 3.7.3

V:\Yaan\learn\test111>where jupyter C:\Users\XinYaanZyoy\Anaconda3\Scripts\jupyter.exe V:\Yaan\learn\test111>jupyter --version 4.4.0

V:\Yaan\learn\test111>where sos C:\Users\XinYaanZyoy\Anaconda3\Scripts\sos.exe V:\Yaan\learn\test111>sos --version sos 0.19.3 for Python 3.7.3

V:\Yaan\learn\test111>where git C:\Program Files\Git\cmd\git.exe V:\Yaan\learn\test111>git --version git version 2.20.1.windows.1

git add./ and git commit -m "testing" sucsessful!!

but V:\Yaan\learn\test111>sos run release.sos clean ERROR: Failed to execute global statement sys.path.append('docs') #begin_parameter pip pip = sos_handle_parameter_('pip', 'pip'

) #end_parameter pip : name 'sys' is not defined

my machine problem or this ipynb-website problme?????

[philomorphism]

okay! i get it! there was a need of "import sys" which i later noticed that is latest commit in release.sos!!

so instead of downloading latest-release-v0.9.5 i downloaded the repo itself and tried to run..

sos run release.sos ERROR: No step to generate target Py_Module("date_parser") requested by default_1

now what's wrong??

[gaow]

@XinYaanZyoy would you mind checking out release v0.9.6 I just made? The issue you are having is due to absence of date_parser packages. The current version will install it for you.

[philomorphism]

[philomorphism]

ohh i forgot to initiating, adding and commiting to git VCS!

i tried again and the erros is....

[gaow]

Okay then it boils down to a Windows specific problem now: we are looking for /bin/bash but cannot find it.

There are workarounds but the simplest would be to make sure you have bash command available from your terminal in the first place?

[philomorphism]

i have Cygwin64 Terminal and i tried from there!

[philomorphism]

i tried in the windows command prompt again and this time the error is....

[gaow]

@XinYaanZyoy what's going on there is that you've got some intermediate _index.ipynb files as relics in your previous failed runs -- you should see them via git status. It is never an issue to me earlier because all my runs are successfully. For now please manually remove them. I'll try to modify the script to skip them if they exist.

My point previously is that you should first make sure in your shell when you type bash it works. In any case, let me try to adjust my script a bit more and get back to you hoping we can address both issues from my end. I do not own a windows machine but I think I can guess what's goingon and fix.

[gaow]

@XinYaanZyoy if you pull the latest master it should solve the bash issue --it will basically try to determine your command shell environment and try to use the approperate interpreter. I am not sure about the _index.ipynb complaint now that I look at my code because they should be skipped already so there should not be any overlap between input and output files. And I cannot reproduce the issue on my end. In any case possibly manually removing them should work. Sorry we dont own or work with Windows machines. This is possibly the best we can do at this point (other than building and releasing the whole thing with docker.)

[philomorphism]

no sir, bash is not working on cmd!!

and i downloaded the latest one but it won't work i think because for that you have to change all those instructions like.... rmdir instead of rm for cmd!!

see...

[philomorphism]

ok let me try to do it by creating docker image with ubuntu and py3.7!

[gaow]

Okay it seems we are one step closer? I just pushed to master replacing rm with Python's os.remove. That should take care of a couple more errors. Not sure if that's enough to get it work.

[philomorphism]

is it possible that i run " sos run release.sos clean" and the next ones from gitlab pipeline, if we use docker image which has python, pip, git, sos, jupyter???

[gaow]

Yes it should work for Windows if we provide a docker image that contains all the resource necessary to run this. I was just trying to remove blockers you've identified to try get away with making a docker image.

[philomorphism]

Congratulations! clean works!

V:\Yaan\learn\test007>sos run release.sos clean INFO: Running clean: INFO: Workflow clean (ID=bee5e6c592ab83e6) is executed successfully with 1 completed step.

---

but...

---

`V:\Yaan\learn\test007>sos run release.sos -s force INFO: Running Configure website:

:54: YAMLLoadWarning: calling yaml.load() without Loader=... is deprecated, as the default Loader is unsafe. Please read https://msg.pyyaml.org/load for full details. :83: YAMLLoadWarning: calling yaml.load() without Loader=... is deprecated, as the default Loader is unsafe. Please read https://msg.pyyaml.org/load for full details. INFO: Running Build website: INFO: Running Index notebook generator: INFO: output: analysis\_index.ipynb setup\_index.ipynb... (5 items) INFO: Running Jupyter HTML template updater: INFO: output: docs\analysis.tpl docs\setup.tpl... (6 items) INFO: Running Index HTML updater: V:\Yaan\learn\test007>jupyter nbconvert setup\index.ipynb --output V:\Yaan\learn\test007\docs\setup.html --template docs/index.tpl V:\Yaan\learn\test007># sed -i 's/ class="prompt input_prompt" 0

In & nbsp;\[[0-9]\]: docs\setup.html 0<\/div 1>//g' V:\Yaan\learn\test007>if [[ 0 -gt 0 ]]; then V:\Yaan\learn\test007>jupyter nbconvert license\index.ipynb --output V:\Yaan\learn\test007\docs\license.html --template docs/index.tpl V:\Yaan\learn\test007># sed -i 's/ class="prompt input_prompt" 0

In & nbsp;\[[0-9]\]: docs\license.html 0<\/div 1>//g' V:\Yaan\learn\test007>if [[ 0 -gt 0 ]]; then V:\Yaan\learn\test007>jupyter nbconvert notes\index.ipynb --output V:\Yaan\learn\test007\docs\notes.html --template docs/index.tpl V:\Yaan\learn\test007># sed -i 's/ class="prompt input_prompt" 0

In & nbsp;\[[0-9]\]: docs\notes.html 0<\/div 1>//g' V:\Yaan\learn\test007>if [[ 0 -gt 0 ]]; then V:\Yaan\learn\test007>jupyter nbconvert protected\_index.ipynb --output V:\Yaan\learn\test007\docs\protected.html --template docs/index.tpl V:\Yaan\learn\test007># sed -i 's/ class="prompt input_prompt" 0

In & nbsp;\[[0-9]\]: docs\protected.html 0<\/div 1>//g' V:\Yaan\learn\test007>if [[ 0 -gt 0 ]]; then V:\Yaan\learn\test007>jupyter nbconvert analysis\index.ipynb --output V:\Yaan\learn\test007\docs\index.html --template docs/index.tpl V:\Yaan\learn\test007># sed -i 's/ class="prompt input_prompt" 0

In & nbsp;\[[0-9]\]: docs\index.html 0<\/div 1>//g' V:\Yaan\learn\test007>if [[ 0 -gt 0 ]]; then ERROR: [9bd5f261-10b4-4a07-a65c-524239784a73]: [update-hp]: Failed to execute .sos/update-hp_0_346a835c.bat exitcode=255, workdir=V:\Yaan\learn\test007` `---------------------------------------------------------------------------` `[update-hp + update-nb + update-wf + upda]: 5 pending steps: update-nb, update-wf, update-toc, update-search, remove-obsolete [default]: 1 pending step: default_4 V:\Yaan\learn\test007>`

gaow commented 5 years ago

Right, it means the core command jupyter nbconvert does not work and there is no way around it. You'd have to either configure Windows properly, or perhaps a lot easier, create a docker image and run from there. Would be nice if you give a stab at the latter. I might find some time to do it later but not promises now.

philomorphism commented 5 years ago

thank you for your responses, i'm now working on the docker solution! will inform details here; either errors or success!

philomorphism commented 5 years ago

(off topic) can you say what i understand is correct or not(it'll help me in writing dockerfile and gitlab-ci.yml)

1. release.sos searches the directories metioned in configure.yml and makes htmls of those ipynbs in it.
2. and then it makes some files in docs directory to make a webpages

so i need write permission from gitlab to github, right?

gaow commented 5 years ago

Not familiar about gitlab (never used it). But your 1&2 are correct. If you decide to do it I'd suggest starting simple -- create Ubuntu virtual machine from your Windows OS, then try to install software in the VM until you are able to get it work. Then whatever you did to setup the Ubuntu VM will end up in your docker, if you take good notes of what you did in Ubuntu. For folder accessibility issues it should be docker parameter -v (for volume) to map directories so you can have access to your disk from docker container instances. But that's the next step.

• © Githubissues.
• Githubissues is a development platform for aggregating issues.

[gaow]

Right, it means the core command jupyter nbconvert does not work and there is no way around it. You'd have to either configure Windows properly, or perhaps a lot easier, create a docker image and run from there. Would be nice if you give a stab at the latter. I might find some time to do it later but not promises now.

[philomorphism]

thank you for your responses, i'm now working on the docker solution! will inform details here; either errors or success!

[philomorphism]

(off topic) can you say what i understand is correct or not(it'll help me in writing dockerfile and gitlab-ci.yml)

1. release.sos searches the directories metioned in configure.yml and makes htmls of those ipynbs in it.
2. and then it makes some files in docs directory to make a webpages

so i need write permission from gitlab to github, right?

[gaow]

Not familiar about gitlab (never used it). But your 1&2 are correct. If you decide to do it I'd suggest starting simple -- create Ubuntu virtual machine from your Windows OS, then try to install software in the VM until you are able to get it work. Then whatever you did to setup the Ubuntu VM will end up in your docker, if you take good notes of what you did in Ubuntu. For folder accessibility issues it should be docker parameter -v (for volume) to map directories so you can have access to your disk from docker container instances. But that's the next step.