search

AktugHakan / NinovaArsivci

Ninova'daki dosyaları toplu bir şekilde indirmek için bir Python programı
13 stars 2 forks source link

The program is not efficient #11

Closed mustafahdurmaz closed 1 year ago

mustafahdurmaz commented 1 year ago

Every time I run the program, it takes a lot of time to download the whole ninova again. Instead, it may be a more effective way to identify and download only newly added files with the help of database. We will also distinguish the newly added files.

OS: Windows 11 Version: Nightly

IsE333 commented 1 year ago

Every time I run the program, it takes a lot of time to download the whole ninova again. Instead, it may be a more effective way to identify and download only newly added files with the help of database. We will also distinguish the newly added files.

OS: Windows 11 Version: Nightly

i do have the same problem too. Totally 300-400mb sized files are downloading everytime the program runs.

AktugHakan commented 1 year ago

It takes time to implement multiprocessing on different OS architectures, I will temporarily disable the multiprocessing on Windows, threading will still work as before.

mustafahdurmaz commented 1 year ago

I expect you to share this update as soon as possible Sir.

AktugHakan commented 1 year ago

I fixed and shared the update, it is currently on Nightly branch.