search

scimma / blast

Django web app for the automatic characterization of supernova hosts
MIT License
1 stars 2 forks source link

Djones sed param docs #246

Closed djones1040 closed 2 months ago

djones1040 commented 3 months ago

Fixes #247

Description of the Change

documentation on SED parameters, plus a more full list of SED parameters in the data model and displayed on the webpages. See also #247 for more information

manning-ncsa commented 3 months ago

@djones1040 Initial launch fails (bash run/blast.up.sh full_dev, open http://localhost:8000) because of the missing latexify package. You'll need to update the Dockerfile(s).

manning-ncsa commented 3 months ago

@djones1040 I shoehorned in the dependency installation for expediency, and after adjusting the new resource limit value for memory usage to 8GB, the global SED task for 2024ozn finally completed on my laptop so that I could verify the display of the new Global parameter details.

Blast_2024ozn_results.pdf

djones1040 commented 2 months ago

hey looks great! no issues I can see. We will have to back-populate this info for the other SNe, but I wrote a script for that that uses the saved output files on disk

manning-ncsa commented 2 months ago

@djones1040 I convoluted this pull request with two other branches that working on this led me to create, each of which have been merged into this SED param docs branch. One branch was a refactoring of the way you locally build and view the documentation such that it is lightweight for ReadtheDocs to build and consistent with the local build. The second branch was a conversion of the way data tables are initially populated; instead of initializing the database with loaddata to load the fixture files, I wrote a migration script that does this in a way that is guaranteed to be schema consistent, with the bonus that it is not redundantly executed each time the application starts. This required me to write a temporary script to hard-code primary key values for each of the records in the fixture files; there was a side-effect of the exporting of the imported/modified fixture data that some of the content was altered superficially in terms of YAML formatting or dropping extraneous trailing zeros from numbers.

manning-ncsa commented 2 months ago

As of commit df5f1e2 the unit tests have been fixed (or disabled) and all pass. There is an outstanding issue with the docs build I will address next.

$ bash run/blast.up.sh ci
Retaining all data volumes.
+ docker compose --profile ci --project-name blast -f docker/docker-compose.yml --env-file env/.env.default --env-file env/.env.ci up --build
[+] Building 0.3s (17/17) FINISHED                                                                                                                                docker:default
 => [app_ci internal] load build definition from Dockerfile                                                                                                                 0.0s
 => => transferring dockerfile: 1.08kB                                                                                                                                      0.0s
 => [app_ci internal] load metadata for docker.io/library/python:3.11                                                                                                       0.2s
 => [app_ci internal] load metadata for hub.ncsa.illinois.edu/blast/deps:20240815                                                                                           0.0s
 => [app_ci internal] load .dockerignore                                                                                                                                    0.0s
 => => transferring context: 2B                                                                                                                                             0.0s
 => [app_ci internal] load build context                                                                                                                                    0.0s
 => => transferring context: 18.70kB                                                                                                                                        0.0s
 => [app_ci stage-1 1/9] FROM docker.io/library/python:3.11@sha256:a23661e4d5dacf56028a800d3af100397a99b120d0f0de5892db61437fd9eb6c                                         0.0s
 => [app_ci deps 1/1] FROM hub.ncsa.illinois.edu/blast/deps:20240815                                                                                                        0.0s
 => CACHED [app_ci stage-1 2/9] RUN apt-get update && DEBIAN_FRONTEND=noninteractive && apt-get install -y --no-install-recommends     ffmpeg     libsm6     libxext6       0.0s
 => CACHED [app_ci stage-1 3/9] COPY ./debug/debug_ipython.py /root/.ipython/profile_default/startup/                                                                       0.0s
 => CACHED [app_ci stage-1 4/9] COPY --from=deps /usr/local/lib/python3.11/site-packages/ /usr/local/lib/python3.11/site-packages/                                          0.0s
 => CACHED [app_ci stage-1 5/9] COPY --from=deps /usr/local/bin/ /usr/local/bin/                                                                                            0.0s
 => CACHED [app_ci stage-1 6/9] RUN git clone https://github.com/manning-ncsa/astro_ghost.git /tmp/blast && cd /tmp/blast &&     git checkout c8b887a006cf5053eda426446ddb  0.0s
 => CACHED [app_ci stage-1 7/9] RUN mkdir /app                                                                                                                              0.0s
 => CACHED [app_ci stage-1 8/9] COPY . /app                                                                                                                                 0.0s
 => CACHED [app_ci stage-1 9/9] WORKDIR /app                                                                                                                                0.0s
 => [app_ci] exporting to image                                                                                                                                             0.0s
 => => exporting layers                                                                                                                                                     0.0s
 => => writing image sha256:c8b4e99e183f5ec43a32d36d56e11f718ae244f15943ae52b3212ec41e859f8c                                                                                0.0s
 => => naming to hub.ncsa.illinois.edu/blast/blast:latest                                                                                                                   0.0s
 => [app_ci] resolving provenance for metadata file                                                                                                                         0.0s
[+] Running 5/5
 ✔ Network blast_default          Created                                                                                                                                   0.1s 
 ✔ Volume "blast_django-static"   Created                                                                                                                                   0.0s 
 ✔ Volume "blast_blast-db"        Created                                                                                                                                   0.0s 
 ✔ Container blast-app_ci-1       Created                                                                                                                                   0.0s 
 ✔ Container blast-database_ci-1  Created                                                                                                                                   0.0s 
Attaching to app_ci-1, database_ci-1
"Object does not exist at path “/org/freedesktop/NetworkManager/ActiveConnection/37”"
app_ci-1       | "/mnt/data/.initializing_data" not found. Running initialization script...
database_ci-1  | 2024-08-16 18:35:35+00:00 [Note] [Entrypoint]: Entrypoint script for MariaDB Server 1:10.11.8+maria~ubu2204 started.
database_ci-1  | 2024-08-16 18:35:36+00:00 [Warn] [Entrypoint]: /sys/fs/cgroup///memory.pressure not writable, functionality unavailable to MariaDB
database_ci-1  | 2024-08-16 18:35:36+00:00 [Note] [Entrypoint]: Switching to dedicated user 'mysql'
database_ci-1  | 2024-08-16 18:35:36+00:00 [Note] [Entrypoint]: Entrypoint script for MariaDB Server 1:10.11.8+maria~ubu2204 started.
database_ci-1  | 2024-08-16 18:35:36+00:00 [Note] [Entrypoint]: Initializing database files
database_ci-1  | 2024-08-16 18:35:36 0 [Warning] mariadbd: io_uring_queue_init() failed with errno 1
database_ci-1  | 2024-08-16 18:35:36 0 [Warning] InnoDB: liburing disabled: falling back to innodb_use_native_aio=OFF
database_ci-1  | 
database_ci-1  | 
database_ci-1  | PLEASE REMEMBER TO SET A PASSWORD FOR THE MariaDB root USER !
database_ci-1  | To do so, start the server, then issue the following command:
database_ci-1  | 
database_ci-1  | '/usr/bin/mariadb-secure-installation'
database_ci-1  | 
database_ci-1  | which will also give you the option of removing the test
database_ci-1  | databases and anonymous user created by default.  This is
database_ci-1  | strongly recommended for production servers.
database_ci-1  | 
database_ci-1  | See the MariaDB Knowledgebase at https://mariadb.com/kb
database_ci-1  | 
database_ci-1  | Please report any problems at https://mariadb.org/jira
database_ci-1  | 
database_ci-1  | The latest information about MariaDB is available at https://mariadb.org/.
database_ci-1  | 
database_ci-1  | Consider joining MariaDB's strong and vibrant community:
database_ci-1  | https://mariadb.org/get-involved/
database_ci-1  | 
database_ci-1  | 2024-08-16 18:35:37+00:00 [Note] [Entrypoint]: Database files initialized
database_ci-1  | 2024-08-16 18:35:37+00:00 [Note] [Entrypoint]: Starting temporary server
database_ci-1  | 2024-08-16 18:35:37+00:00 [Note] [Entrypoint]: Waiting for server startup
database_ci-1  | 2024-08-16 18:35:37 0 [Note] Starting MariaDB 10.11.8-MariaDB-ubu2204 source revision 3a069644682e336e445039e48baae9693f9a08ee as process 98
database_ci-1  | 2024-08-16 18:35:37 0 [Note] InnoDB: Compressed tables use zlib 1.2.11
database_ci-1  | 2024-08-16 18:35:37 0 [Note] InnoDB: Number of transaction pools: 1
database_ci-1  | 2024-08-16 18:35:37 0 [Note] InnoDB: Using crc32 + pclmulqdq instructions
database_ci-1  | 2024-08-16 18:35:37 0 [Note] mariadbd: O_TMPFILE is not supported on /tmp (disabling future attempts)
database_ci-1  | 2024-08-16 18:35:37 0 [Warning] mariadbd: io_uring_queue_init() failed with errno 1
database_ci-1  | 2024-08-16 18:35:37 0 [Warning] InnoDB: liburing disabled: falling back to innodb_use_native_aio=OFF
database_ci-1  | 2024-08-16 18:35:37 0 [Note] InnoDB: Initializing buffer pool, total size = 128.000MiB, chunk size = 2.000MiB
database_ci-1  | 2024-08-16 18:35:37 0 [Note] InnoDB: Completed initialization of buffer pool
database_ci-1  | 2024-08-16 18:35:37 0 [Note] InnoDB: File system buffers for log disabled (block size=512 bytes)
database_ci-1  | 2024-08-16 18:35:37 0 [Note] InnoDB: End of log at LSN=45502
database_ci-1  | 2024-08-16 18:35:37 0 [Note] InnoDB: 128 rollback segments are active.
database_ci-1  | 2024-08-16 18:35:37 0 [Note] InnoDB: Setting file './ibtmp1' size to 12.000MiB. Physically writing the file full; Please wait ...
database_ci-1  | 2024-08-16 18:35:37 0 [Note] InnoDB: File './ibtmp1' size is now 12.000MiB.
database_ci-1  | 2024-08-16 18:35:37 0 [Note] InnoDB: log sequence number 45502; transaction id 14
database_ci-1  | 2024-08-16 18:35:37 0 [Note] Plugin 'FEEDBACK' is disabled.
database_ci-1  | 2024-08-16 18:35:37 0 [Warning] 'user' entry 'root@761dfd21315c' ignored in --skip-name-resolve mode.
database_ci-1  | 2024-08-16 18:35:37 0 [Warning] 'proxies_priv' entry '@% root@761dfd21315c' ignored in --skip-name-resolve mode.
database_ci-1  | 2024-08-16 18:35:37 0 [Note] mariadbd: ready for connections.
database_ci-1  | Version: '10.11.8-MariaDB-ubu2204'  socket: '/run/mysqld/mysqld.sock'  port: 0  mariadb.org binary distribution
database_ci-1  | 2024-08-16 18:35:38+00:00 [Note] [Entrypoint]: Temporary server started.
database_ci-1  | 2024-08-16 18:35:39+00:00 [Note] [Entrypoint]: Creating database blast_db
database_ci-1  | 2024-08-16 18:35:39+00:00 [Note] [Entrypoint]: Securing system users (equivalent to running mysql_secure_installation)
database_ci-1  | 
database_ci-1  | 2024-08-16 18:35:39+00:00 [Note] [Entrypoint]: Stopping temporary server
database_ci-1  | 2024-08-16 18:35:39 0 [Note] mariadbd (initiated by: unknown): Normal shutdown
database_ci-1  | 2024-08-16 18:35:39 0 [Note] InnoDB: FTS optimize thread exiting.
database_ci-1  | 2024-08-16 18:35:39 0 [Note] InnoDB: Starting shutdown...
database_ci-1  | 2024-08-16 18:35:39 0 [Note] InnoDB: Dumping buffer pool(s) to /var/lib/mysql/ib_buffer_pool
database_ci-1  | 2024-08-16 18:35:39 0 [Note] InnoDB: Buffer pool(s) dump completed at 240816 18:35:39
database_ci-1  | 2024-08-16 18:35:39 0 [Note] InnoDB: Removed temporary tablespace data file: "./ibtmp1"
database_ci-1  | 2024-08-16 18:35:39 0 [Note] InnoDB: Shutdown completed; log sequence number 47092; transaction id 15
database_ci-1  | 2024-08-16 18:35:39 0 [Note] mariadbd: Shutdown complete
database_ci-1  | 
database_ci-1  | 2024-08-16 18:35:39+00:00 [Note] [Entrypoint]: Temporary server stopped
database_ci-1  | 
database_ci-1  | 2024-08-16 18:35:39+00:00 [Note] [Entrypoint]: MariaDB init process done. Ready for start up.
database_ci-1  | 
database_ci-1  | 2024-08-16 18:35:39 0 [Note] Starting MariaDB 10.11.8-MariaDB-ubu2204 source revision 3a069644682e336e445039e48baae9693f9a08ee as process 1
database_ci-1  | 2024-08-16 18:35:39 0 [Note] InnoDB: Compressed tables use zlib 1.2.11
database_ci-1  | 2024-08-16 18:35:39 0 [Note] InnoDB: Number of transaction pools: 1
database_ci-1  | 2024-08-16 18:35:39 0 [Note] InnoDB: Using crc32 + pclmulqdq instructions
database_ci-1  | 2024-08-16 18:35:39 0 [Note] mariadbd: O_TMPFILE is not supported on /tmp (disabling future attempts)
database_ci-1  | 2024-08-16 18:35:39 0 [Warning] mariadbd: io_uring_queue_init() failed with errno 1
database_ci-1  | 2024-08-16 18:35:39 0 [Warning] InnoDB: liburing disabled: falling back to innodb_use_native_aio=OFF
database_ci-1  | 2024-08-16 18:35:39 0 [Note] InnoDB: Initializing buffer pool, total size = 128.000MiB, chunk size = 2.000MiB
database_ci-1  | 2024-08-16 18:35:39 0 [Note] InnoDB: Completed initialization of buffer pool
database_ci-1  | 2024-08-16 18:35:39 0 [Note] InnoDB: File system buffers for log disabled (block size=512 bytes)
database_ci-1  | 2024-08-16 18:35:39 0 [Note] InnoDB: End of log at LSN=47092
database_ci-1  | 2024-08-16 18:35:39 0 [Note] InnoDB: 128 rollback segments are active.
database_ci-1  | 2024-08-16 18:35:39 0 [Note] InnoDB: Setting file './ibtmp1' size to 12.000MiB. Physically writing the file full; Please wait ...
database_ci-1  | 2024-08-16 18:35:39 0 [Note] InnoDB: File './ibtmp1' size is now 12.000MiB.
database_ci-1  | 2024-08-16 18:35:39 0 [Note] InnoDB: log sequence number 47092; transaction id 16
database_ci-1  | 2024-08-16 18:35:39 0 [Note] InnoDB: Loading buffer pool(s) from /var/lib/mysql/ib_buffer_pool
database_ci-1  | 2024-08-16 18:35:39 0 [Note] Plugin 'FEEDBACK' is disabled.
database_ci-1  | 2024-08-16 18:35:40 0 [Warning] You need to use --log-bin to make --expire-logs-days or --binlog-expire-logs-seconds work.
database_ci-1  | 2024-08-16 18:35:40 0 [Note] InnoDB: Buffer pool(s) load completed at 240816 18:35:39
database_ci-1  | 2024-08-16 18:35:40 0 [Note] Server socket created on IP: '0.0.0.0'.
database_ci-1  | 2024-08-16 18:35:40 0 [Note] Server socket created on IP: '::'.
database_ci-1  | 2024-08-16 18:35:40 0 [Note] mariadbd: ready for connections.
database_ci-1  | Version: '10.11.8-MariaDB-ubu2204'  socket: '/run/mysqld/mysqld.sock'  port: 3306  mariadb.org binary distribution
app_ci-1       | INFO: Required data files pass integrity check.
app_ci-1       | Data initialization complete.
app_ci-1       | wait-for-it.sh: waiting for database:3306 without a timeout
database_ci-1  | 2024-08-16 18:35:43 3 [Warning] Aborted connection 3 to db: 'unconnected' user: 'unauthenticated' host: '172.18.0.3' (This connection closed normally without authentication)
app_ci-1       | wait-for-it.sh: database:3306 is available after 0 seconds
app_ci-1       | "/mnt/data/.initializing_db" not found. Running initialization script...
app_ci-1       | 2024-08-16 18:35:47.361447: E external/local_xla/xla/stream_executor/cuda/cuda_fft.cc:485] Unable to register cuFFT factory: Attempting to register factory for plugin cuFFT when one has already been registered
app_ci-1       | 2024-08-16 18:35:47.383255: E external/local_xla/xla/stream_executor/cuda/cuda_dnn.cc:8454] Unable to register cuDNN factory: Attempting to register factory for plugin cuDNN when one has already been registered
app_ci-1       | 2024-08-16 18:35:47.391988: E external/local_xla/xla/stream_executor/cuda/cuda_blas.cc:1452] Unable to register cuBLAS factory: Attempting to register factory for plugin cuBLAS when one has already been registered
app_ci-1       | 2024-08-16 18:35:47.409533: I tensorflow/core/platform/cpu_feature_guard.cc:210] This TensorFlow binary is optimized to use available CPU instructions in performance-critical operations.
app_ci-1       | To enable the following instructions: AVX2 FMA, in other operations, rebuild TensorFlow with the appropriate compiler flags.
app_ci-1       | 2024-08-16 18:35:48.520695: W tensorflow/compiler/tf2tensorrt/utils/py_utils.cc:38] TF-TRT Warning: Could not find TensorRT
app_ci-1       | Operations to perform:
app_ci-1       |   Apply all migrations: admin, auth, contenttypes, django_celery_beat, django_celery_results, django_cron, host, sessions, silk
app_ci-1       | Running migrations:
app_ci-1       |   Applying contenttypes.0001_initial... OK
app_ci-1       |   Applying auth.0001_initial... OK
app_ci-1       |   Applying admin.0001_initial... OK
app_ci-1       |   Applying admin.0002_logentry_remove_auto_add... OK
app_ci-1       |   Applying admin.0003_logentry_add_action_flag_choices... OK
app_ci-1       |   Applying contenttypes.0002_remove_content_type_name... OK
app_ci-1       |   Applying auth.0002_alter_permission_name_max_length... OK
app_ci-1       |   Applying auth.0003_alter_user_email_max_length... OK
app_ci-1       |   Applying auth.0004_alter_user_username_opts... OK
app_ci-1       |   Applying auth.0005_alter_user_last_login_null... OK
app_ci-1       |   Applying auth.0006_require_contenttypes_0002... OK
app_ci-1       |   Applying auth.0007_alter_validators_add_error_messages... OK
app_ci-1       |   Applying auth.0008_alter_user_username_max_length... OK
app_ci-1       |   Applying auth.0009_alter_user_last_name_max_length... OK
app_ci-1       |   Applying auth.0010_alter_group_name_max_length... OK
app_ci-1       |   Applying auth.0011_update_proxy_permissions... OK
app_ci-1       |   Applying auth.0012_alter_user_first_name_max_length... OK
app_ci-1       |   Applying django_celery_beat.0001_initial... OK
app_ci-1       |   Applying django_celery_beat.0002_auto_20161118_0346... OK
app_ci-1       |   Applying django_celery_beat.0003_auto_20161209_0049... OK
app_ci-1       |   Applying django_celery_beat.0004_auto_20170221_0000... OK
app_ci-1       |   Applying django_celery_beat.0005_add_solarschedule_events_choices... OK
app_ci-1       |   Applying django_celery_beat.0006_auto_20180322_0932... OK
app_ci-1       |   Applying django_celery_beat.0007_auto_20180521_0826... OK
app_ci-1       |   Applying django_celery_beat.0008_auto_20180914_1922... OK
app_ci-1       |   Applying django_celery_beat.0006_auto_20180210_1226... OK
app_ci-1       |   Applying django_celery_beat.0006_periodictask_priority... OK
app_ci-1       |   Applying django_celery_beat.0009_periodictask_headers... OK
app_ci-1       |   Applying django_celery_beat.0010_auto_20190429_0326... OK
app_ci-1       |   Applying django_celery_beat.0011_auto_20190508_0153... OK
app_ci-1       |   Applying django_celery_beat.0012_periodictask_expire_seconds... OK
app_ci-1       |   Applying django_celery_beat.0013_auto_20200609_0727... OK
app_ci-1       |   Applying django_celery_beat.0014_remove_clockedschedule_enabled... OK
app_ci-1       |   Applying django_celery_beat.0015_edit_solarschedule_events_choices... OK
app_ci-1       |   Applying django_celery_beat.0016_alter_crontabschedule_timezone... OK
app_ci-1       |   Applying django_celery_beat.0017_alter_crontabschedule_month_of_year... OK
app_ci-1       |   Applying django_celery_beat.0018_improve_crontab_helptext... OK
app_ci-1       |   Applying django_celery_results.0001_initial... OK
app_ci-1       |   Applying django_celery_results.0002_add_task_name_args_kwargs... OK
app_ci-1       |   Applying django_celery_results.0003_auto_20181106_1101... OK
app_ci-1       |   Applying django_celery_results.0004_auto_20190516_0412... OK
app_ci-1       |   Applying django_celery_results.0005_taskresult_worker... OK
app_ci-1       |   Applying django_celery_results.0006_taskresult_date_created... OK
app_ci-1       |   Applying django_celery_results.0007_remove_taskresult_hidden... OK
app_ci-1       |   Applying django_celery_results.0008_chordcounter... OK
app_ci-1       |   Applying django_celery_results.0009_groupresult... OK
app_ci-1       |   Applying django_celery_results.0010_remove_duplicate_indices... OK
app_ci-1       |   Applying django_celery_results.0011_taskresult_periodic_task_name... OK
app_ci-1       |   Applying django_cron.0001_initial... OK
app_ci-1       |   Applying django_cron.0002_remove_max_length_from_CronJobLog_message... OK
app_ci-1       |   Applying django_cron.0003_cronjoblock... OK
app_ci-1       |   Applying host.0001_initial... OK
app_ci-1       |   Applying host.0002_auto_20220601_1914... OK
app_ci-1       |   Applying host.0003_acknowledgement... OK
app_ci-1       |   Applying host.0004_filter_kcorrect_name... OK
app_ci-1       |   Applying host.0005_prospectorresult_host... OK
app_ci-1       |   Applying host.0006_host_milkyway_dust_reddening... OK
app_ci-1       |   Applying host.0007_transient_processing_status... OK
app_ci-1       |   Applying host.0008_auto_20220822_2335... OK
app_ci-1       |   Applying host.0009_alter_aperturephotometry_flux... OK
app_ci-1       |   Applying host.0010_auto_20221030_1810... OK
app_ci-1       |   Applying host.0011_filter_magnitude_zero_point_keyword... OK
app_ci-1       |   Applying host.0012_filter_ab_offset... OK
app_ci-1       |   Applying host.0013_sedfittingresult_mass_surviving_ratio... OK
app_ci-1       |   Applying host.0014_taskregister_user_warning... OK
app_ci-1       |   Applying host.0015_alter_taskregister_user_warning... OK
app_ci-1       |   Applying host.0016_transient_added_by... OK
app_ci-1       |   Applying host.0017_alter_aperturephotometry_is_validated... OK
app_ci-1       |   Applying host.0018_auto_20240205_0240... OK
app_ci-1       |   Applying host.0019_cutout_message... OK
app_ci-1       |   Applying host.0020_transient_progress... OK
app_ci-1       |   Applying host.0021_alter_transient_progress... OK
app_ci-1       |   Applying host.0022_externalrequest... OK
app_ci-1       |   Applying host.0023_alter_transient_name... OK
app_ci-1       |   Applying host.0024_load_fixtures... OK
app_ci-1       |   Applying host.0025_sedfittingresult_dust1_fraction_16_and_more... OK
app_ci-1       |   Applying sessions.0001_initial... OK
app_ci-1       |   Applying silk.0001_initial... OK
app_ci-1       |   Applying silk.0002_auto_update_uuid4_id_field... OK
app_ci-1       |   Applying silk.0003_request_prof_file... OK
app_ci-1       |   Applying silk.0004_request_prof_file_storage... OK
app_ci-1       |   Applying silk.0005_increase_request_prof_file_length... OK
app_ci-1       |   Applying silk.0006_fix_request_prof_file_blank... OK
app_ci-1       |   Applying silk.0007_sqlquery_identifier... OK
app_ci-1       |   Applying silk.0008_sqlquery_analysis... OK
app_ci-1       | 
app_ci-1       | 265 static files copied to '/app/static'.
app_ci-1       | Django database initialization complete.
app_ci-1       | 2024-08-16 18:35:58.415247: E external/local_xla/xla/stream_executor/cuda/cuda_fft.cc:485] Unable to register cuFFT factory: Attempting to register factory for plugin cuFFT when one has already been registered
app_ci-1       | 2024-08-16 18:35:58.432772: E external/local_xla/xla/stream_executor/cuda/cuda_dnn.cc:8454] Unable to register cuDNN factory: Attempting to register factory for plugin cuDNN when one has already been registered
app_ci-1       | 2024-08-16 18:35:58.437719: E external/local_xla/xla/stream_executor/cuda/cuda_blas.cc:1452] Unable to register cuBLAS factory: Attempting to register factory for plugin cuBLAS when one has already been registered
app_ci-1       | 2024-08-16 18:35:58.449643: I tensorflow/core/platform/cpu_feature_guard.cc:210] This TensorFlow binary is optimized to use available CPU instructions in performance-critical operations.
app_ci-1       | To enable the following instructions: AVX2 FMA, in other operations, rebuild TensorFlow with the appropriate compiler flags.
app_ci-1       | 2024-08-16 18:36:00.034905: W tensorflow/compiler/tf2tensorrt/utils/py_utils.cc:38] TF-TRT Warning: Could not find TensorRT
app_ci-1       | Found 56 test(s).
app_ci-1       | Creating test database for alias 'default' ('test_blast_db')...
app_ci-1       | Operations to perform:
app_ci-1       |   Synchronize unmigrated apps: api, bootstrap4, crispy_bootstrap4, crispy_forms, django_filters, django_tables2, messages, rest_framework, revproxy, staticfiles, users
app_ci-1       |   Apply all migrations: admin, auth, contenttypes, django_celery_beat, django_celery_results, django_cron, host, sessions, silk
app_ci-1       | Synchronizing apps without migrations:
app_ci-1       |   Creating tables...
app_ci-1       |     Running deferred SQL...
app_ci-1       | Running migrations:
app_ci-1       |   Applying contenttypes.0001_initial... OK
app_ci-1       |   Applying auth.0001_initial... OK
app_ci-1       |   Applying admin.0001_initial... OK
app_ci-1       |   Applying admin.0002_logentry_remove_auto_add... OK
app_ci-1       |   Applying admin.0003_logentry_add_action_flag_choices... OK
app_ci-1       |   Applying contenttypes.0002_remove_content_type_name... OK
app_ci-1       |   Applying auth.0002_alter_permission_name_max_length... OK
app_ci-1       |   Applying auth.0003_alter_user_email_max_length... OK
app_ci-1       |   Applying auth.0004_alter_user_username_opts... OK
app_ci-1       |   Applying auth.0005_alter_user_last_login_null... OK
app_ci-1       |   Applying auth.0006_require_contenttypes_0002... OK
app_ci-1       |   Applying auth.0007_alter_validators_add_error_messages... OK
app_ci-1       |   Applying auth.0008_alter_user_username_max_length... OK
app_ci-1       |   Applying auth.0009_alter_user_last_name_max_length... OK
app_ci-1       |   Applying auth.0010_alter_group_name_max_length... OK
app_ci-1       |   Applying auth.0011_update_proxy_permissions... OK
app_ci-1       |   Applying auth.0012_alter_user_first_name_max_length... OK
app_ci-1       |   Applying django_celery_beat.0001_initial... OK
app_ci-1       |   Applying django_celery_beat.0002_auto_20161118_0346... OK
app_ci-1       |   Applying django_celery_beat.0003_auto_20161209_0049... OK
app_ci-1       |   Applying django_celery_beat.0004_auto_20170221_0000... OK
app_ci-1       |   Applying django_celery_beat.0005_add_solarschedule_events_choices... OK
app_ci-1       |   Applying django_celery_beat.0006_auto_20180322_0932... OK
app_ci-1       |   Applying django_celery_beat.0007_auto_20180521_0826... OK
app_ci-1       |   Applying django_celery_beat.0008_auto_20180914_1922... OK
app_ci-1       |   Applying django_celery_beat.0006_auto_20180210_1226... OK
app_ci-1       |   Applying django_celery_beat.0006_periodictask_priority... OK
app_ci-1       |   Applying django_celery_beat.0009_periodictask_headers... OK
app_ci-1       |   Applying django_celery_beat.0010_auto_20190429_0326... OK
app_ci-1       |   Applying django_celery_beat.0011_auto_20190508_0153... OK
app_ci-1       |   Applying django_celery_beat.0012_periodictask_expire_seconds... OK
app_ci-1       |   Applying django_celery_beat.0013_auto_20200609_0727... OK
app_ci-1       |   Applying django_celery_beat.0014_remove_clockedschedule_enabled... OK
app_ci-1       |   Applying django_celery_beat.0015_edit_solarschedule_events_choices... OK
app_ci-1       |   Applying django_celery_beat.0016_alter_crontabschedule_timezone... OK
app_ci-1       |   Applying django_celery_beat.0017_alter_crontabschedule_month_of_year... OK
app_ci-1       |   Applying django_celery_beat.0018_improve_crontab_helptext... OK
app_ci-1       |   Applying django_celery_results.0001_initial... OK
app_ci-1       |   Applying django_celery_results.0002_add_task_name_args_kwargs... OK
app_ci-1       |   Applying django_celery_results.0003_auto_20181106_1101... OK
app_ci-1       |   Applying django_celery_results.0004_auto_20190516_0412... OK
app_ci-1       |   Applying django_celery_results.0005_taskresult_worker... OK
app_ci-1       |   Applying django_celery_results.0006_taskresult_date_created... OK
app_ci-1       |   Applying django_celery_results.0007_remove_taskresult_hidden... OK
app_ci-1       |   Applying django_celery_results.0008_chordcounter... OK
app_ci-1       |   Applying django_celery_results.0009_groupresult... OK
app_ci-1       |   Applying django_celery_results.0010_remove_duplicate_indices... OK
app_ci-1       |   Applying django_celery_results.0011_taskresult_periodic_task_name... OK
app_ci-1       |   Applying django_cron.0001_initial... OK
app_ci-1       |   Applying django_cron.0002_remove_max_length_from_CronJobLog_message... OK
app_ci-1       |   Applying django_cron.0003_cronjoblock... OK
app_ci-1       |   Applying host.0001_initial... OK
app_ci-1       |   Applying host.0002_auto_20220601_1914... OK
app_ci-1       |   Applying host.0003_acknowledgement... OK
app_ci-1       |   Applying host.0004_filter_kcorrect_name... OK
app_ci-1       |   Applying host.0005_prospectorresult_host... OK
app_ci-1       |   Applying host.0006_host_milkyway_dust_reddening... OK
app_ci-1       |   Applying host.0007_transient_processing_status... OK
app_ci-1       |   Applying host.0008_auto_20220822_2335... OK
app_ci-1       |   Applying host.0009_alter_aperturephotometry_flux... OK
app_ci-1       |   Applying host.0010_auto_20221030_1810... OK
app_ci-1       |   Applying host.0011_filter_magnitude_zero_point_keyword... OK
app_ci-1       |   Applying host.0012_filter_ab_offset... OK
app_ci-1       |   Applying host.0013_sedfittingresult_mass_surviving_ratio... OK
app_ci-1       |   Applying host.0014_taskregister_user_warning... OK
app_ci-1       |   Applying host.0015_alter_taskregister_user_warning... OK
app_ci-1       |   Applying host.0016_transient_added_by... OK
app_ci-1       |   Applying host.0017_alter_aperturephotometry_is_validated... OK
app_ci-1       |   Applying host.0018_auto_20240205_0240... OK
app_ci-1       |   Applying host.0019_cutout_message... OK
app_ci-1       |   Applying host.0020_transient_progress... OK
app_ci-1       |   Applying host.0021_alter_transient_progress... OK
app_ci-1       |   Applying host.0022_externalrequest... OK
app_ci-1       |   Applying host.0023_alter_transient_name... OK
app_ci-1       |   Applying host.0024_load_fixtures... OK
app_ci-1       |   Applying host.0025_sedfittingresult_dust1_fraction_16_and_more... OK
app_ci-1       |   Applying sessions.0001_initial... OK
app_ci-1       |   Applying silk.0001_initial... OK
app_ci-1       |   Applying silk.0002_auto_update_uuid4_id_field... OK
app_ci-1       |   Applying silk.0003_request_prof_file... OK
app_ci-1       |   Applying silk.0004_request_prof_file_storage... OK
app_ci-1       |   Applying silk.0005_increase_request_prof_file_length... OK
app_ci-1       |   Applying silk.0006_fix_request_prof_file_blank... OK
app_ci-1       |   Applying silk.0007_sqlquery_identifier... OK
app_ci-1       |   Applying silk.0008_sqlquery_analysis... OK
app_ci-1       | System check identified no issues (0 silenced).
app_ci-1       | <photutils.segmentation.core.SegmentationImage>
app_ci-1       | shape: (1384, 1384)
app_ci-1       | nlabels: 83
app_ci-1       | labels: [ 1  2  3  4  5 ... 79 80 81 82 83]
app_ci-1       | test_aperture_construction (host.tests.test_aperture_construction.TestApertureConstruction.test_aperture_construction) ... ok
app_ci-1       | test_aperture_failures (host.tests.test_aperture_construction.TestApertureConstruction.test_aperture_failures) ... ok
app_ci-1       | test_cutout_download (host.tests.test_cutouts.CutoutDownloadTest.test_cutout_download)
app_ci-1       | <SkyCoord (ICRS): (ra, dec) in deg
app_ci-1       |     (355.53628555, 48.70907059)>
app_ci-1       | https://dr12.sdss.org/fields/raDec?ra=355.53628555&dec=48.70907059166666
app_ci-1       | Could not download SDSS_i data
app_ci-1       | exception: list index out of range
app_ci-1       | <SkyCoord (ICRS): (ra, dec) in deg
app_ci-1       |     (355.53628555, 48.70907059)>
app_ci-1       | https://dr12.sdss.org/fields/raDec?ra=355.53628555&dec=48.70907059166666
app_ci-1       | Could not download SDSS_r data
app_ci-1       | exception: list index out of range
app_ci-1       | <SkyCoord (ICRS): (ra, dec) in deg
app_ci-1       |     (355.53628555, 48.70907059)>
app_ci-1       | https://dr12.sdss.org/fields/raDec?ra=355.53628555&dec=48.70907059166666
app_ci-1       | Could not download SDSS_u data
app_ci-1       | exception: list index out of range
app_ci-1       | <SkyCoord (ICRS): (ra, dec) in deg
app_ci-1       |     (355.53628555, 48.70907059)>
app_ci-1       | https://dr12.sdss.org/fields/raDec?ra=355.53628555&dec=48.70907059166666
app_ci-1       | Could not download SDSS_z data
app_ci-1       | exception: list index out of range
app_ci-1       | <SkyCoord (ICRS): (ra, dec) in deg
app_ci-1       |     (355.53628555, 48.70907059)>
app_ci-1       | https://dr12.sdss.org/fields/raDec?ra=355.53628555&dec=48.70907059166666
app_ci-1       | Could not download SDSS_g data
app_ci-1       | exception: list index out of range
app_ci-1       | " ... ok
app_ci-1       | test_mwebv_transient (host.tests.test_ebv.EBVTest.test_mwebv_transient) ... ok
app_ci-1       | Couldn't find where you want to save GHOST. Setting location to package path...
app_ci-1       | Successfully created dummy database.
app_ci-1       | 
app_ci-1       | 1 transients not found in GHOST by name, trying a coordinate search...
app_ci-1       | 1 transients not found in GHOST by name or coordinates, manually associating...
app_ci-1       | Successfully read in GWGC catalog.
app_ci-1       | 
app_ci-1       | Found 1 hosts in GLADE! See gladeDLR.txt for details.
app_ci-1       | Found matches for 100.0% of events.
app_ci-1       | Saving table of hosts to /data/ghost_outputtransients_20240816183815102195/tables/FinalAssociationTable.csv.
app_ci-1       | 0 transients found by name, 0 transients found by coordinates, 1 transients manually associated.
app_ci-1       | Dust map data directory "/data/ghost_data/dust_model/sfddata-master" already exists.
app_ci-1       | photo-z weights file "/data/ghost_data/photoz_model/MLP_lupton.hdf5" already exists.
app_ci-1       | warning : photo-z step failed: DustMap query failed because the expected coordinates didntexist in DF, likely the match of any Hosts into PanStarrs failed
app_ci-1       | test_aperture_construction (host.tests.test_host_match.TestHostMatch.test_aperture_construction) ... ok
app_ci-1       | test_dec_string (host.tests.test_models.SkyObjectTest.test_dec_string) ... ok
app_ci-1       | test_ra_string (host.tests.test_models.SkyObjectTest.test_ra_string) ... /usr/local/lib/python3.11/site-packages/django/db/models/base.py:366: RuntimeWarning: Model 'host.test_obj' was already registered. Reloading models is not advised as it can lead to inconsistencies, most notably with related models.
app_ci-1       |   new_class._meta.apps.register_model(new_class._meta.app_label, new_class)
app_ci-1       | ok
app_ci-1       | test_sky_coord (host.tests.test_models.SkyObjectTest.test_sky_coord) ... ok
app_ci-1       | test_global_aperture_photometry (host.tests.test_photometry.TestValidatePhotometry.test_global_aperture_photometry) ... ok
app_ci-1       | test_local_aperture_redshifts (host.tests.test_photometry.TestValidatePhotometry.test_local_aperture_redshifts) ... ok
app_ci-1       | <photutils.segmentation.core.SegmentationImage>
app_ci-1       | shape: (1384, 1384)
app_ci-1       | nlabels: 77
app_ci-1       | labels: [ 1  2  3  4  5 ... 73 74 75 76 77]
app_ci-1       | <photutils.segmentation.core.SegmentationImage>
app_ci-1       | shape: (1384, 1384)
app_ci-1       | nlabels: 77
app_ci-1       | labels: [ 1  2  3  4  5 ... 73 74 75 76 77]
app_ci-1       | test_validate_global_photometry (host.tests.test_photometry.TestValidatePhotometry.test_validate_global_photometry) ... ok
app_ci-1       | test_validate_local_photometry (host.tests.test_photometry.TestValidatePhotometry.test_validate_local_photometry) ... ok
app_ci-1       | test_failed_status (host.tests.test_processing.GHOSTRunnerTest.test_failed_status) ... ok
app_ci-1       | test_prereqs (host.tests.test_processing.GHOSTRunnerTest.test_prereqs) ... ok
app_ci-1       | test_failed_status (host.tests.test_processing.ImageDownloadTest.test_failed_status) ... ok
app_ci-1       | test_prereqs (host.tests.test_processing.ImageDownloadTest.test_prereqs) ... ok
app_ci-1       | task_register_item: TaskRegister object (37)
app_ci-1       | test_run_process (host.tests.test_processing.ImageDownloadTest.test_run_process) ... ok
app_ci-1       | test_task_register_init (host.tests.test_processing.InitializeTaskRegisterTest.test_task_register_init) ... ok
app_ci-1       | test_create_object (host.tests.test_processing.TaskRunnerTest.test_create_object) ... ok
app_ci-1       | test_no_register_items (host.tests.test_processing.TaskRunnerTest.test_no_register_items) ... ok
app_ci-1       | test_overwrite_object (host.tests.test_processing.TaskRunnerTest.test_overwrite_object) ... ok
app_ci-1       | test_run_not_processed (host.tests.test_processing.TaskRunnerTest.test_run_not_processed) ... ok
app_ci-1       | test_update_status_last_modified (host.tests.test_processing.TaskRunnerTest.test_update_status_last_modified) ... ok
app_ci-1       | test_update_status_status_change (host.tests.test_processing.TaskRunnerTest.test_update_status_status_change) ... ok
app_ci-1       | test_task_type (host.tests.test_processing.TestAllRegisteredTaskRunners.test_task_type) ... ok
app_ci-1       | test_transient_task_failed_status (host.tests.test_processing.TestAllRegisteredTaskRunners.test_transient_task_failed_status) ... ok
app_ci-1       | test_transient_task_name (host.tests.test_processing.TestAllRegisteredTaskRunners.test_transient_task_name) ... ok
app_ci-1       | test_transient_task_prerequisites (host.tests.test_processing.TestAllRegisteredTaskRunners.test_transient_task_prerequisites) ... ok
app_ci-1       | test_filter_conversion (host.tests.test_sedfitting.FilterTest.test_filter_conversion)
app_ci-1       | Test blast filter is converted to sedpy filter ... ok
app_ci-1       | test_build_obs (host.tests.test_sedfitting.PropsectorBuildObsTest.test_build_obs) ... ok
app_ci-1       | test_marker (host.tests.test_transient_name_server.BuildTNSHeaderTest.test_marker) ... ok
app_ci-1       | test_return_contains_correct_key (host.tests.test_transient_name_server.BuildTNSHeaderTest.test_return_contains_correct_key) ... ok
app_ci-1       | test_return_dictionary_size (host.tests.test_transient_name_server.BuildTNSHeaderTest.test_return_dictionary_size) ... ok
app_ci-1       | test_return_type_is_dict (host.tests.test_transient_name_server.BuildTNSHeaderTest.test_return_type_is_dict) ... ok
app_ci-1       | test_get_mode (host.tests.test_transient_name_server.BuildTNSUrlTest.test_get_mode) ... ok
app_ci-1       | test_invalid_mode (host.tests.test_transient_name_server.BuildTNSUrlTest.test_invalid_mode) ... ok
app_ci-1       | test_search_mode (host.tests.test_transient_name_server.BuildTNSUrlTest.test_search_mode) ... ok
app_ci-1       | test_tns_to_blast (host.tests.test_transient_name_server.ConvertTNSToBLASTTest.test_tns_to_blast) ... ok
app_ci-1       | test_catalog_download (host.tests.test_utils.CatalogDownloadTest.test_catalog_download)
app_ci-1       | Test that catalog data can be downloaded. ... ok
app_ci-1       | test_filter_information_download (host.tests.test_utils.CatalogDownloadTest.test_filter_information_download)
app_ci-1       | Test that catalog filter data can be downloaded. ... ok
app_ci-1       | test_tansient_page (host.tests.test_views.SEDPlotTest.test_tansient_page) ... ok
app_ci-1       | test_transient_list_page (host.tests.test_views.ViewTest.test_transient_list_page) ... ok
app_ci-1       | test_transient_page (host.tests.test_views.ViewTest.test_transient_page) ... ok
app_ci-1       | test_get_no_transient (api.tests.test_api.APITest.test_get_no_transient) ... ok
app_ci-1       | test_transient_already_in_database (api.tests.test_api.APITest.test_transient_already_in_database) ... ok
app_ci-1       | test_transient_bad_post (api.tests.test_api.APITest.test_transient_bad_post) ... ok
app_ci-1       | test_transient_get (api.tests.test_api.APITest.test_transient_get) ... ok
app_ci-1       | test_transient_post (api.tests.test_api.APITest.test_transient_post) ... ok
app_ci-1       | test_all_datamodel_components_output_type (api.tests.test_datamodel.DataModelComponentTests.test_all_datamodel_components_output_type) ... ok
app_ci-1       | test_aperture_build (api.tests.test_datamodel.DatamodelConstructionTest.test_aperture_build) ... ok
app_ci-1       | test_datamodel_build_with_data (api.tests.test_datamodel.DatamodelConstructionTest.test_datamodel_build_with_data) ... ok
app_ci-1       | test_datamodel_build_without_data (api.tests.test_datamodel.DatamodelConstructionTest.test_datamodel_build_without_data) ... ok
app_ci-1       | test_photometry_build (api.tests.test_datamodel.DatamodelConstructionTest.test_photometry_build) ... ok
app_ci-1       | test_sed_fitting_result_build (api.tests.test_datamodel.DatamodelConstructionTest.test_sed_fitting_result_build) ... ok
app_ci-1       | test_component_group_unpacking (api.tests.test_datamodel.GroupComponentUnpackTest.test_component_group_unpacking) ... ok
app_ci-1       | test_login (users.tests.test_user_login.LogInTest.test_login) ... ok
app_ci-1       | 
app_ci-1       | ----------------------------------------------------------------------
app_ci-1       | Ran 56 tests in 169.619s
app_ci-1       | 
app_ci-1       | OK
app_ci-1       | Destroying test database for alias 'default' ('test_blast_db')...
app_ci-1       | /usr/local/lib/python3.11/site-packages/coverage/report_core.py:115: CoverageWarning: Couldn't parse '/app/config-3.py': No source for code: '/app/config-3.py'. (couldnt-parse)
app_ci-1       |   coverage._warn(msg, slug="couldnt-parse")
app_ci-1       | /usr/local/lib/python3.11/site-packages/coverage/report_core.py:115: CoverageWarning: Couldn't parse '/app/config.py': No source for code: '/app/config.py'. (couldnt-parse)
app_ci-1       |   coverage._warn(msg, slug="couldnt-parse")
app_ci-1       | Name                              Stmts   Miss  Cover
app_ci-1       | -----------------------------------------------------
app_ci-1       | api/__init__.py                       0      0   100%
app_ci-1       | api/apps.py                           4      0   100%
app_ci-1       | api/components.py                    28      0   100%
app_ci-1       | api/datamodel.py                     26      0   100%
app_ci-1       | api/serializers.py                   51      1    98%
app_ci-1       | api/tests/__init__.py                 0      0   100%
app_ci-1       | api/tests/test_api.py                89      0   100%
app_ci-1       | api/tests/test_datamodel.py         103      0   100%
app_ci-1       | api/urls.py                           9      1    89%
app_ci-1       | api/views.py                        138      0   100%
app_ci-1       | host/base_tasks.py                  174     40    77%
app_ci-1       | host/catalog_photometry.py           23      3    87%
app_ci-1       | host/cutouts.py                     262    112    57%
app_ci-1       | host/forms.py                        15      0   100%
app_ci-1       | host/ghost.py                        30      4    87%
app_ci-1       | host/host_utils.py                  265     81    69%
app_ci-1       | host/managers.py                     31      1    97%
app_ci-1       | host/models.py                      310     27    91%
app_ci-1       | host/photometric_calibration.py      23      7    70%
app_ci-1       | host/plotting_utils.py              196     78    60%
app_ci-1       | host/postprocess_prosp.py           128    109    15%
app_ci-1       | host/prospector.py                  325    275    15%
app_ci-1       | host/system_tasks.py                181    114    37%
app_ci-1       | host/tables.py                       20      2    90%
app_ci-1       | host/tasks.py                        41     24    41%
app_ci-1       | host/transient_name_server.py       134     98    27%
app_ci-1       | host/transient_tasks.py             407    153    62%
app_ci-1       | host/views.py                       232     94    59%
app_ci-1       | host/workflow.py                     44     19    57%
app_ci-1       | users/__init__.py                     0      0   100%
app_ci-1       | users/apps.py                         4      0   100%
app_ci-1       | users/tests/__init__.py               0      0   100%
app_ci-1       | users/tests/test_user_login.py        9      0   100%
app_ci-1       | users/urls.py                         3      0   100%
app_ci-1       | -----------------------------------------------------
app_ci-1       | TOTAL                              3305   1243    62%
app_ci-1       | /usr/local/lib/python3.11/site-packages/coverage/report_core.py:115: CoverageWarning: Couldn't parse '/app/config-3.py': No source for code: '/app/config-3.py'. (couldnt-parse)
app_ci-1       |   coverage._warn(msg, slug="couldnt-parse")
app_ci-1       | /usr/local/lib/python3.11/site-packages/coverage/report_core.py:115: CoverageWarning: Couldn't parse '/app/config.py': No source for code: '/app/config.py'. (couldnt-parse)
app_ci-1       |   coverage._warn(msg, slug="couldnt-parse")
app_ci-1       | Wrote XML report to coverage.xml
app_ci-1 exited with code 0