hotosm / osm-fieldwork

Processing field data from ODK to OpenStreetMap format, and other field data collection utils.
GNU Affero General Public License v3.0
16 stars 77 forks source link

Ignore DB files in pyproject.toml build opts #109

Open spwoodcock opened 1 year ago

spwoodcock commented 1 year ago

During development it is useful to pip install odkconvert from a local directory.

If the user has downloaded files into the directory, e.g. a taginfo database, then the pip install takes a long time (as it includes GBs of data).

In the tool.pdm.build section for pyproject.toml we should have an excludes wildcard for .db and .sql files.

The install should then be tested to see if packaging time is still minimal.

mohammadareeb95 commented 1 year ago

exclude = ["*.db", "*.sql"]

Then pip install -e . --no-cache-dir to prevent it from using any cached files from previous installations. This will ensure that the package is built from scratch using the modified pyproject.toml file.

I think this should fix the issue. What do you think @spwoodcock ?

robsavoye commented 1 year ago

The smartest thing to do is not drop the huge taginfo database files file in the source tree, where pip tries to install them. which is what I unfortunately did. :-)

spwoodcock commented 1 year ago

exclude = ["*.db", "*.sql"]

Then pip install -e . --no-cache-dir to prevent it from using any cached files from previous installations. This will ensure that the package is built from scratch using the modified pyproject.toml file.

I think this should fix the issue. What do you think @spwoodcock ?

That syntax would only exclude files in the root directory. In standard wildcard syntax we would need something like **.*.sql. However, I'm not certain if this is supported as docs for setuptools and pdm are a bit sparse.

In this case I would exclude from the odkconvert directory, where the scripts are:

excludes = ["odkconvert/*.db", "odkconvert/*.sql"]