search

sobrus / FastLacellsGenerator

Fast lacells.db database generator script for LocalGSMBackend by n76
GNU General Public License v3.0
44 stars 10 forks source link

Script hangs when OCID is not available #15

Closed bege10 closed 3 years ago

bege10 commented 3 years ago

Hello, I use this script in WSL (Ubuntu for Windows) and it works well (OCID and Mozilla) if the downloads succeed. But when OCID does not provide data because the last download is less than 24 hrs ago the script fails or hangs if download argument -q has been removed.

With the -q argument the script stops with

gzip: stdin: not in gzip format.

Without the -q argument

  1. it tries to download from OCID what fails as expected and creates a 0B ocid.csv file
  2. it displays:

    gzip: stdin: not in gzip format

  3. it downloads Mozilla and creates a correct mozilla.csv
  4. and then the script hangs, it neither continues nor stops.

I have little knowledge of bash scripting but as far as I understand the script in this case it should unzip the ocid.csv.gz from a previous download. (KEEP_FILES is set to 1.) Is that the file the script says about that it is not in gzip format? But it has been compressed by the script in a previous download.

Eventually in the temp folder there are all necessary files

Does the script not work correct in this case or did I misunderstand something?

sobrus commented 3 years ago

Hi!

Unfortunately, I was unable to reproduce this freeze issue on my machine (Manjaro Linux). It may be WSL specific problem, but I don't have any windows machine right now, and I've never used WSL myself.

That said, I've found a problem with wget error handling - the internal functions were abnormally terminated on wget failure (and backup files were not used). The script basically waits for all download functions to finish (wait commands before actual processing). Wget failure may cause it to wait forever in such case on WSL but not on my linux (but I'm just guessing now).

So it may have caused your problems. I've fixed it, so please try to test the latest script version and let me know if it helps. The "gzip: stdin: not in gzip format" is wget error message, it will be still displayed, even if backup file is used instead.

bege10 commented 3 years ago

Sorry for the long delay. I ran the fixed script several times shortly after each other using OCID and Mozilla and all runs created the lacells.db correctly. Thank you very much!