train autoencoder fails

[AuritaP]

I just installed the program and began using it. I think I installed correctly in my windows 10, 64bit machine and I got as far as creating a project, doing the make patches, but when I tried to do the train model, it failed. I am using just three images at 5x each, to try the software first. They come from a whole slide image, but I just took certain parts of the slide to make those three images and see how the application perform. I have not been able to find anything about that error. The error appears quite quickly after I tell it to train. Can you orient me?

[tasvora]

Hi Aurita

Sorry you are having issues, will you please be able to send in a couple of files from the application folder. 1) QA.log, jobs.log and flask.log 2) data.db ( they will all have generated in your QuickAnnotator folder from where you initiated the application) And can you do a pip freeze of your environment and attach that as well. Some more information needed what version of python are you using, what is the version of your cuda

[AuritaP]

Thank you very much for the prompt response. It may be that it is all my fault since my knowledge of programming is very limited and I stumbled installing the program . I have to try several things until I was able to install everything that was required and some of that several times. Instead of going directly to the pip3 installation, I did other things that I thought I needed. I used anaconda to install python in my computer, I installed the cuda toolkit verison 10, (BUT I do not have an nvidia card, so I am not sure that installation is ok), I had to re-install pytorch several times since it look like not all of the dependencies were being installed, until finally I used pip3 install -r requirements.txt. That worked AFTER I took the version number for numpy, scikit_image, scikit_learn, and opencd_contrib_python_headless. When I tried to install it with the versions that you have, the installation said those version were not compatible and advise me to let the program make its choices, so I took the versions out and it installed with no errors. So I ran it and got stuck when it tried to do the learning. The version of python that i have installed is 3.9.7, some how I can not get what version of pytorch is installed but when I installed, I did the checked that it installed correctly. BUT I could not figure out how to tell it to install pytorch for a NO_CUDA installation, the instructions on hoe to do it, did not work. Now I found a different page with slightly different syntax for how to install pytorch for NO CUDA system, but have not tried that yet. I am sending the files that you requested, will wait for your response but I am thinking that maybe I should uninstall everything and start new, but will wait for your response. I tried to send the data.db file, but it did not let me flask.log jobs.log QA.log .

[AuritaP]

I just learned hoe to do pip freeze so here is the file that i sent the results to troubleshoot.txt

[tasvora]

Hi

CUDA will not work with out NVIDIA. And this tool is tested with python 3.8 on Chrome Browser You can try with the current version of python but if it still does not work you will have to downgrade to 3.8

Since you do not have CUDA, you can use the cpu version of pytorch pip3 install torch torchvision torchaudio

But before you do this , can you run this command

 pip freeze

And copy all the output and send it in a file.

[AuritaP]

yes, I just send you the output of the pip in another comment above

[AuritaP]

i just labeled troubleshoot

[tasvora]

You need to uninstall torch==1.8.1+cu111 torchaudio==0.8.1 torchvision==0.9.1+cu111

and then install the cpu version . restart the application , create a new project and follow the steps https://github.com/choosehappy/QuickAnnotator/wiki/User-Manual

Regards Tasneem

[AuritaP]

thank you very much, I will try that

[AuritaP]

I am sorry to bother you again, but I see that people have trouble uninstalling torch, can you tell me the syntax to follow?

[tasvora]

pip uninstall torch==1.8.1+cu111 pip uninstall torchaudio==0.8.1 pip uninstall torchvision==0.9.1+cu111

try this

[AuritaP]

will try now

[AuritaP]

well it said that they all have been uninstall now, will follow with the other steps now

[AuritaP]

well that did not work. I have now installed chrome and followed your steps above and have the same problem. I will now try to change the version of python to 3.8. Any advise on the best way to downgrade python? Should I have to unisntall anaconda and begin with a new installation?

[AuritaP]

Searching on how to do it, it looks like all that I have to do is type conda install python=3.8.0 without having to uninstall the prior version?

[tasvora]

Some details about what did not work ? Did pytorch uninstall failed ? Did the cpu version of pytorch install failed ?

You can have two versions of python on your machine, but you need to use the correct version for this application.

Regards Tasneem

[AuritaP]

Sorry, I should have explain better. Pytorch un-installation went well, at least no errors were seen and it said it did what I asked to do in each of the three cases. The cpu installation of pytorch proceeded with no problems. Close all of the windows, re-started again. I also installed chrome browser in my computer. Deleted old project, created a new one and use your test images for the tubules segments, was able to run the first part of the program successfully (the make patches). The program stalled when I tried to do the train autoencoder , same place where it stalled the last time. Will proceed to install the other version of python BUT HOW do I tell the program to use that version?

[AuritaP]

and this is the pip freeze output after getting rid of unwanted ptorch versions and installing the cpu version afterPtorchCPUinstalled.txt

[AuritaP]

IT WORKED, well after two more tries. I created a new environmental variable to install python 3.8 and when I checked the version it says that 3.8.12 was installed, so to be sure I created another variable and forced it to install 3.8.0 . But of course I mess up the installation of the program again in that environment so before coming back here, decided to go back to the 3.8.12 version environment and install the program there. Since now I know that the three torch related dependencies in your requirements file did not work for my NON coda version, I took them out of the file, but of course when I went to ran pip3 with it, I used the ole file, so I had to uninstall them and then go and install the right torch version. And now the program has been running for a while doing the retrain in your sample images. So I will now get to learn the program and see if it can do what I need it to do. Thank you very much for your help. I am only entering this here to help somebody with my little knowledge. The only thing i need to do either install or learn to use the whole image slider so I can use it in my slides

[AuritaP]

OK Now I have a new problem. I am trying to use cli to divide my images, but the program is not running at all, it says that there is no module named open slide when I ran the command python extract_titles....... input_pattern f:..... -o f:..... when I ran the help command after pyhton and extract ...... , get the same error when i ran the command alone, it says it can open the file, it can not find the directory.

[choosehappy]

if you're using whole slide images, you'll need to install openslide as well, both the c-library and the associate python library

the instructions for doing that are available on the openslide website here: https://openslide.org/download/

On Fri, Apr 22, 2022 at 9:22 PM AuritaP @.***> wrote:

OK Now I have a new problem. I am trying to use cli to divide my images, but the program is not running at all, it says that there is no module named open slide when I ran the command python extract_titles....... input_pattern f:..... -o f:..... when I ran the help command after pyhton and extract ...... , get the same error when i ran the command alone, it says it can open the file, it can not find the directory.

— Reply to this email directly, view it on GitHub https://github.com/choosehappy/QuickAnnotator/issues/24#issuecomment-1106793938, or unsubscribe https://github.com/notifications/unsubscribe-auth/ACJ3XTGSSOL5R7ITQYXLYA3VGL4BDANCNFSM5UAI5FMQ . You are receiving this because you are subscribed to this thread.Message ID: @.***>

[AuritaP]

Thank you very much. I will look into that Aurita

Sent from my iPhone

On Apr 23, 2022, at 3:09 PM, choosehappy @.***> wrote:

 if you're using whole slide images, you'll need to install openslide as well, both the c-library and the associate python library

the instructions for doing that are available on the openslide website here: https://openslide.org/download/

On Fri, Apr 22, 2022 at 9:22 PM AuritaP @.***> wrote:

OK Now I have a new problem. I am trying to use cli to divide my images, but the program is not running at all, it says that there is no module named open slide when I ran the command python extract_titles....... input_pattern f:..... -o f:..... when I ran the help command after pyhton and extract ...... , get the same error when i ran the command alone, it says it can open the file, it can not find the directory.

— Reply to this email directly, view it on GitHub https://github.com/choosehappy/QuickAnnotator/issues/24#issuecomment-1106793938, or unsubscribe https://github.com/notifications/unsubscribe-auth/ACJ3XTGSSOL5R7ITQYXLYA3VGL4BDANCNFSM5UAI5FMQ . You are receiving this because you are subscribed to this thread.Message ID: @.***>

— Reply to this email directly, view it on GitHub, or unsubscribe. You are receiving this because you authored the thread.

[AuritaP]

ok I have installed the python library in the right environment and have downloaded the c library for open slide for windows and unpack that, but not sure how to proceed. I mean how to install that library

[tasvora]

To use openslide with python , you will also need the openslide package installed

pip install openslide-python

And then you can follow this link and configure openslide library DLL's to be available for the python program https://lists.andrew.cmu.edu/pipermail/openslide-users/2014-November/000948.html

And then write program in python

import openslide

print("Import Done")

What is the output do you get any error ? or do you see the print message

[AuritaP]

SOLVED. After trying several things, the only thing that worked was changing a line inside the lowlevel.py file that is located in the bin directory of openslide. I added the exact location of the liboenslide-0-dll. Somebody recommend this and it worked for them and me. The line as I changed is below _lib = cdll.LoadLibrary('F:\Users\Aura\anaconda3\envs\py38\openslide-win64-20171122\bin\libopenslide-0.dll')

I was able to install open slide after that and began dividing some of my ndpi images. Now I will begin to familiarize myself with the program and see if I can trained to do what I need it to do. Thank you for your assistance. You may close this issue.

[choosehappy]

if that was your issue, you could also add a line like this before importing openslide:

import os
os.environ['PATH'] = 'C:\\research\\openslide\\bin' + ';' + os.environ['PATH'] #can either specify openslide bin path in PATH, or add it dynamically

but if you do this type of work with high frequency, its better to set the windows path variable to point at the openslide directory, something like this:

https://www.computerhope.com/issues/ch000549.htm