search

mrjleo / boilernet

Boilerplate Removal using Deep Learning
MIT License
82 stars 18 forks source link

Problem with running with class weight #6

Closed michaelshekasta closed 3 years ago

michaelshekasta commented 3 years ago

Hi,

First of all, thanks for sharing your code. There are many packages that you missed like bs4 and downloading staff of NLTK. When I tried to run the code (training phase), I got an error: image

I found that class weight does not represent well (as np array instead of dictionary). When I change it into the dictionary I got this error: image

Regards, Michael

mrjleo commented 3 years ago

Hi, could you elaborate? Besides bs4 and NLTK (neither of which are missing from readme or the requirements.txt), what is missing?

Regarding your problem, are you sure you have the correct versions of all packages installed?

michaelshekasta commented 3 years ago
  1. only bs4 and NLTK (as I remembered)
  2. Nope, tensorflow=2.4.0, I also try to run sample_weight but I can't due to using tesorflow files ("Cannot use "sample_weight" in tf.data.Dataset") - I can't install tensorflow=2.1.0 probably due to python version (3.8.2)
mrjleo commented 3 years ago

In that case I recommend using Anaconda, I think you should be able to install the correct libraries this way.

michaelshekasta commented 3 years ago

which python version? which anaconda version?

mrjleo commented 3 years ago

Just check the readme please. It's all there.

michaelshekasta commented 3 years ago

Now it's work with that version, but pay attention - when I changed the class weight as a dictionary (as I mentioned before, I got the same error). I am suspect that the class_weight option with np array is a bug. please take a look at this thread.

mrjleo commented 3 years ago

Thanks, it looks like the TF/Keras documentation indeed changed (from array to dict). However I think the way it's done now works properly in TF 2.1. However I did not test it yet. If you find it is indeed a bug, please feel free to open another issue.