export_studies crashed: django.http.response.Http404: No records found matching [Study.id=12]

[lparsons]

BUG DESCRIPTION

Problem

This might be a data issue, since I ran into it running this on tracebase-dev, but I figured I'd record it here now while I investigate.

Steps to reproduce

1. ssh tracebase-dev
2. sudo -iu tracebase
3. cd /var/www/tracebase
4. source /usr/local/tracebase/bin/activate
5. python manage.py export_studies --outdir ~/tracebase-export

Current behavior

WARNING: Failed search to display field fctemplate search conversion of query object for field=value: [id='12']. No records found matching Study.
Traceback (most recent call last):
  File "/var/www/tracebase/manage.py", line 22, in <module>
    main()
  File "/var/www/tracebase/manage.py", line 18, in main
    execute_from_command_line(sys.argv)
  File "/usr/local/tracebase/lib/python3.9/site-packages/django/core/management/__init__.py", line 442, in execute_from_command_line
    utility.execute()
  File "/usr/local/tracebase/lib/python3.9/site-packages/django/core/management/__init__.py", line 436, in execute
    self.fetch_command(subcommand).run_from_argv(self.argv)
  File "/usr/local/tracebase/lib/python3.9/site-packages/django/core/management/base.py", line 412, in run_from_argv
    self.execute(*args, **cmd_options)
  File "/usr/local/tracebase/lib/python3.9/site-packages/django/core/management/base.py", line 458, in execute
    output = self.handle(*args, **options)
  File "/var/www/tracebase/DataRepo/management/commands/export_studies.py", line 39, in handle
    se.export()
  File "/var/www/tracebase/DataRepo/utils/studies_exporter.py", line 90, in export
    qry = self.sg.createNewBasicQuery(
  File "/var/www/tracebase/DataRepo/formats/dataformat_group.py", line 769, in createNewBasicQuery
    dfld, dval = self.searchFieldToDisplayField(mdl, fld, val, qry)
  File "/var/www/tracebase/DataRepo/formats/dataformat_group.py", line 415, in searchFieldToDisplayField
    raise Http404(f"No records found matching [{mdl}.{fld}={val}].")
django.http.response.Http404: No records found matching [Study.id=12].

Expected behavior

Expected all data to be exported to output directory.

Suggested Change

None provided

Comment

None

---

ISSUE OWNER SECTION

Assumptions

1. List of assumptions made WRT the code
2. E.g. We will assume input is correct (explaining why there is no validation)

Limitations

1. A list of things this work will specifically not do
2. E.g. This feature will only handle the most frequent use case X

Affected Components

• change: File path or DB table ...
• add: Environment variable or server setting
• delete: External executable or cron job

Requirements

• [ ] 1. List of numbered conditions to be met for the feature
• [ ] 2. E.g. Every column/row must display a value, i.e. cannot be empty
• [ ] 3. Numbers for reference & checkboxes for progress tracking

DESIGN

GUI Change description

None provided

Code Change Description

None provided

Tests

• [ ] 1. A description of at least one test for each requirement above.
• [ ] 2. E.g. Test for req 2 that there's an exception when display value is ''
• [ ] 3. Numbers for reference & checkboxes for progress tracking

[hepcat72]

@lparsons - Could you look in the log and append the output of this warning to the issue?:

f"WARNING: Failed search to display field {fmt} search conversion of query object for field=value: [{fld}='{val}']. No records found matching {mdl}."

[lparsons]

@hepcat72 Sorry, it was there, but if formatted the comment poorly. Fixed it in the issue.

[hepcat72]

Thanks. Yeah, I had already reproduced it on the website. This search_basic link reproduces the error:

https://tracebase-dev.princeton.edu/DataRepo/search_basic/Study/id/iexact/12/fcirc/

I suspect a dataformat config issue. I should have it worked out soon.

[hepcat72]

Note that the other formats work...

https://tracebase-dev.princeton.edu/DataRepo/search_basic/Study/id/iexact/12/peakgroups/ https://tracebase-dev.princeton.edu/DataRepo/search_basic/Study/id/iexact/12/peakdata/

[lparsons]

Thanks. Yeah, I had already reproduced it on the website. This search_basic link reproduces the error:

https://tracebase-dev.princeton.edu/DataRepo/search_basic/Study/id/iexact/12/fcirc/

I suspect a dataformat config issue. I should have it worked out soon.

Nice! Thanks Rob. Please fix it on the release_2.0.x branch.

[hepcat72]

OK. No. I think this is a data issue. You were right. This is the root of the problem (I did this in the shell on tracebase-dev):

In [33]: FCirc.objects.filter(serum_sample__animal__studies__id__exact=12)
Out[33]: <QuerySet []>

It's an edge case.

My immediate guess (though I haven't looked) is that there is no FCirc data available for that study. Looking now.

[hepcat72]

And yes, that appears to be the case...

So my next question is... was there any accucor data loaded for that study? If so, were there serum samples included?

[lparsons]

Here is the study data: https://github.com/PrincetonUniversity/tracebase-rabinowitz-data/tree/main/tca_flux_healthy_tissues

[hepcat72]

The error comes from the conversion of the Study ID to the study name via FCirc.serum_sample.animal.study. If there are no serum sample, the ID to name conversion fails with that exception. So... This case where there are no serum sample records should be accounted for, because the error makes it seem like the study doesn't exist... which is true in the context of coming from the FCirc table.

[hepcat72]

Here is the study data: https://github.com/PrincetonUniversity/tracebase-rabinowitz-data/tree/main/tca_flux_healthy_tissues

Yes, and in that entire table, there are no serum samples. That's the edge case.

[hepcat72]

Merge both in the release and main branches