search

archesproject / arches

Arches is a web platform for creating, managing, & visualizing geospatial data. Arches was inspired by the needs of the Cultural Heritage community, particularly the widespread need of organizations to build & manage cultural heritage inventories
GNU Affero General Public License v3.0
210 stars 139 forks source link

Fix ResourceWarnings from unclosed files #11091

Closed jacobtylerwalls closed 4 days ago

jacobtylerwalls commented 1 week ago

Running the tests with ResourceWarning enabled flags many instances of unclosed file handles, which is a performance consideration.

We should rewite these open calls as with open(...:

Reproducer

python3 -Wall::ResourceWarning manage.py test tests --settings=tests.test_settings --verbosity=2

Results

Items like:

test_csv_export (tests.exporter.resource_export_tests.BusinessDataExportTests.test_csv_export) ... /Users/jwalls/prj/arches/arches/app/utils/data_management/resources/importer.py:111: ResourceWarning: unclosed file <_io.TextIOWrapper name='tests/fixtures/data/csv/resource_export_test.mapping' mode='r' encoding='UTF-8'>
  self.mapping = json.load(open(path, "r"))
ResourceWarning: Enable tracemalloc to get the object allocation traceback
/Users/jwalls/prj/arches/tests/exporter/resource_export_tests.py:73: ResourceWarning: unclosed file <_io.TextIOWrapper name='tests/fixtures/data/csv/resource_export_test.csv' mode='r' encoding='utf-8'>
  BusinessDataImporter(
ResourceWarning: Enable tracemalloc to get the object allocation traceback
/Users/jwalls/prj/arches/arches/app/utils/data_management/resources/formats/csvfile.py:73: ResourceWarning: unclosed file <_io.TextIOWrapper name='tests/fixtures/data/csv/resource_export_test.mapping' mode='r' encoding='UTF-8'>
  resource_export_configs = json.load(open(configs, "r"))
ResourceWarning: Enable tracemalloc to get the object allocation traceback
/Users/jwalls/prj/arches/tests/exporter/resource_export_tests.py:87: ResourceWarning: unclosed file <_io.TextIOWrapper name='tests/fixtures/data/csv/resource_export_test.csv' mode='r' encoding='utf-8'>
  csv_input = list(
ResourceWarning: Enable tracemalloc to get the object allocation traceback
...
Mark0047 commented 1 week ago

Hi, can I work on this?

chiatt commented 1 week ago

Hi @HarisMughal - you're definitely welcome to work on this.

jacobtylerwalls commented 1 week ago

Hey @HarisMughal :wave: thanks for the message. When getting started, I'd suggest branching off of dev/7.6.x instead of master.

jacobtylerwalls commented 4 days ago

Fixed in #11108