search

rhayes777 / PyAutoFit

PyAutoFit: Classy Probabilistic Programming
https://pyautofit.readthedocs.io/
MIT License
59 stars 10 forks source link

Scrape not working on autolens grid search #924

Closed Jammy2211 closed 6 months ago

Jammy2211 commented 6 months ago

The following script:

https://github.com/Jammy2211/autolens_workspace_test/blob/main/database/build_bug.py

Prints the following output, which leads to database use issues afterwards:

2024-02-08 09:11:24,977 - autofit.database.aggregator.scrape - WARNING - Failed to load samples for lens_0_no_subhalo

By removing the except in the relevent part of PyAutoFit I get this error:

Aggregator loading search_outputs... could take some time.

 A total of 4 search_outputs and results were found.
2024-02-08 09:10:21,308 - autofit.database.aggregator.scrape - INFO - Scraping directory output/base
2024-02-08 09:10:21,309 - autofit.database.aggregator.scrape - INFO - 4 searches found

2024-02-08 09:10:26,535 - autofit.database.aggregator.scrape - INFO - Creating fit for: lens_0_no_subhalo lens_0_no_subhalo subhalo[2]_mass[total]_source_subhalo[search_lens_plane]/2ccbf0421d90c0686c845700ad73bbdc/galaxies_subhalo_mass_centre_0_-3.00_0.00_galaxies_subhalo_mass_centre_1_-3.00_0.00 739d35ca205ee95bb0f4749787693700

2024-02-08 09:10:28,233 - autofit.database.aggregator.scrape - INFO - Creating fit for: lens_0_no_subhalo lens_0_no_subhalo subhalo[2]_mass[total]_source_subhalo[search_lens_plane]/2ccbf0421d90c0686c845700ad73bbdc/galaxies_subhalo_mass_centre_0_-3.00_0.00_galaxies_subhalo_mass_centre_1_0.00_3.00 64f5d2e111db037060c5b7cd2651ca8c

2024-02-08 09:10:29,795 - autofit.database.aggregator.scrape - INFO - Creating fit for: lens_0_no_subhalo lens_0_no_subhalo subhalo[2]_mass[total]_source_subhalo[search_lens_plane]/2ccbf0421d90c0686c845700ad73bbdc/galaxies_subhalo_mass_centre_0_0.00_3.00_galaxies_subhalo_mass_centre_1_-3.00_0.00 bd53cd219567c76d86b6b9a91f5e45eb

2024-02-08 09:10:31,372 - autofit.database.aggregator.scrape - INFO - Creating fit for: lens_0_no_subhalo lens_0_no_subhalo subhalo[2]_mass[total]_source_subhalo[search_lens_plane]/2ccbf0421d90c0686c845700ad73bbdc/galaxies_subhalo_mass_centre_0_0.00_3.00_galaxies_subhalo_mass_centre_1_0.00_3.00 f9213210ebcd8722c7e242bb25036572
Traceback (most recent call last):
  File "/mnt/c/Users/Jammy/Code/PyAuto/autolens_workspace_test/database/build_bug.py", line 51, in <module>
    agg.add_directory(directory=path.join("output", database_name))
  File "/mnt/c/Users/Jammy/Code/PyAuto/PyAutoFit/autofit/database/aggregator/aggregator.py", line 428, in add_directory
    scraper.scrape()
  File "/mnt/c/Users/Jammy/Code/PyAuto/PyAutoFit/autofit/database/aggregator/scrape.py", line 54, in scrape
    for grid_search in self._grid_searches():
  File "/mnt/c/Users/Jammy/Code/PyAuto/PyAutoFit/autofit/database/aggregator/scrape.py", line 122, in _grid_searches
    _add_files(grid_search, item)
  File "/mnt/c/Users/Jammy/Code/PyAuto/PyAutoFit/autofit/database/aggregator/scrape.py", line 159, in _add_files
    fit.samples = item.samples
  File "/mnt/c/Users/Jammy/Code/PyAuto/PyAutoFit/autofit/database/model/fit.py", line 268, in samples
    EfficientSamples(samples),
  File "/mnt/c/Users/Jammy/Code/PyAuto/PyAutoFit/autofit/non_linear/samples/efficient.py", line 19, in __init__
    self.model = samples.model
AttributeError: 'NoneType' object has no attribute 'model'
rhayes777 commented 6 months ago

No search_outputs found in output/base

How do I generate the required output data?

Jammy2211 commented 6 months ago

Missing output/base folder added.