Strain order not updating as expected

[anwarMZ]

For a time-series-based production instance, default_reference_strain_order.json doesn't work well. I wrote a script to order the files based on the latest to earliest in terms of time. However, the order is not reflected on the instance even if the image is rebuilt and the container is restarted.

The contents of the default_reference_strain_order.json file for time-series data are as follows -

[
  "2024-04-09_2024-04-15",
  "2024-04-02_2024-04-08",
  "2024-03-26_2024-04-01",
  "2024-03-19_2024-03-25",
  "2024-03-12_2024-03-18",
  "2024-03-05_2024-03-11",
  "2024-02-27_2024-03-04",
  "2024-02-20_2024-02-26",
  "2024-02-13_2024-02-19",
  "2024-02-06_2024-02-12",
  "2024-01-30_2024-02-05"
] 

the current view of the instance is here

[ivansg44]

Seems to be in reverse order

[anwarMZ]

I am setting up the wastewater instance as I type. Maybe that will further clarify.

[ivansg44]

The reverse order also happens to be the ASCII order. This means the GVF sample names extracted from the header are not being matched to the key in strain_order_dict, in data_parser.py.

[anwarMZ]

I tried this same on the wastewater instance and it looks like the order was again not parsed well.

cat default_reference_strain_order.json

[
  "sample_1_T1",
  "sample_2_T1",
  "sample_3_T1",
  "sample_4_T1",
  "sample_5_T1",
  "sample_6_T1",
  "sample_7_T1",
  "sample_8_T1",
  "sample_9_T1",
  "sample_10_T1",
  "sample_11_T1",
  "sample_12_T1",
  "sample_13_T1",
  "sample_14_T1",
  "sample_15_T1",
  "sample_16_T1",
  "sample_17_T1",
  "sample_18_T1",
  "sample_19_T1",
  "sample_20_T1"
]

the current view of the instance is here

[ivansg44]

It is again in ASCII order

[ivansg44]

Seems to be working fine now. Did you clear the cache directory before rebuilding?

[ivansg44]

It seems the strain order does not update unless you clear the cache directory AND restart the application. But we want it to change whenever the strain order file is modified. The hidden strains file is probably affected similarly. Also, this is likely not related to the docker configuration. It happens if you start the program locally too.

[ivansg44]

I actually think we shouldn't change anything for local development, and just remember after updating the strain order/hidden strains files to 1) clear the cache directory, and 2) restart the application without generating new cache. I think that's sufficient because shouldn't expect users to muck around with the underlying asset files, which would duplicate the issue for them. For docker deployment purposes, implementing #202 should prevent the issue from happening again in the future anyways. We will not include the local cache directory when building the docker image.