search

lmb-embrapa / machado

This repository provides users with a framework to store, search and visualize biological data.
GNU General Public License v3.0
26 stars 15 forks source link

Issue while loading data into the database #333

Closed guff95 closed 2 years ago

guff95 commented 2 years ago

Setup

Python version: 3.8.12 Operating system: Linux

python 3.8.12 (default, Sep 10 2021, 00:16:05) [GCC 7.5.0] CPython Linux-5.4.0-91-generic-x86_64-with-glibc2.27

Issue

when trying to run manage.py runserver it is also working fine. But when I try to load the fasta file into the database with the command python manage.py load_fasta --file organism_chrs.fa --soterm chromosome --organism 'Arabidopsis thaliana' it shows error

 self.cvterm_contained_in = Cvterm.objects.get(
 File "/home/guffi/Documents/projects/whole_genome_ingestion/machado_main/env_mogambo/lib/python3.8/site-packages    /Django-3.2.10-py3.8.egg/django/db/models/manager.py", line 85, in manager_method
  return getattr(self.get_queryset(), name)(*args, **kwargs)
  File "/home/guffi/Documents/projects/whole_genome_ingestion/machado_main/env_mogambo/lib/python3.8/site-packages  /Django-3.2.10-py3.8.egg/django/db/models/query.py", line 435, in get
raise self.model.DoesNotExist( machado.models.DoesNotExist: Cvterm matching query does not exist.)

I tried to do migrations but it shows

return executor(sql, params, many, context)
File "/home/guffi/Documents/projects/whole_genome_ingestion/machado_main/env_mogambo/lib/python3.8/site-packages/Django- 3.2.10-py3.8.egg/django/db/backends/utils.py", line 84, in _execute
 return self.cursor.execute(sql, params)
 File "/home/guffi/Documents/projects/whole_genome_ingestion/machado_main/env_mogambo/lib/python3.8/site-packages/Django-3.2.10-py3.8.egg/django/db/utils.py", line 90, in __exit__
  raise dj_exc_value.with_traceback(traceback) from exc_value
  File "/home/guffi/Documents/projects/whole_genome_ingestion/machado_main/env_mogambo/lib/python3.8/site-packages   /Django-3.2.10-py3.8.egg/django/db/backends/utils.py", line 82, in _execute
  return self.cursor.execute(sql)
  django.db.utils.ProgrammingError: relation "acquisition" already exists

Please help to solve this issue

azneto commented 2 years ago

This error indicates the soterm chromosome is not loaded to the database. Have you loaded the ontologies?

Em sex., 7 de jan. de 2022 03:59, guff95 @.***> escreveu:

Setup

Python version: 3.8.12 Operating system: Linux

python 3.8.12 (default, Sep 10 2021, 00:16:05) [GCC 7.5.0] CPython Linux-5.4.0-91-generic-x86_64-with-glibc2.27 Issue

when trying to run manage.py runserver it is also working fine. But when I try to load the fasta file into the database with the command python manage.py load_fasta --file organism_chrs.fa --soterm chromosome --organism 'Arabidopsis thaliana' it shows error

self.cvterm_contained_in = Cvterm.objects.get( File "/home/guffi/Documents/projects/whole_genome_ingestion/machado_main/env_mogambo/lib/python3.8/site-packages /Django-3.2.10-py3.8.egg/django/db/models/manager.py", line 85, in manager_method return getattr(self.get_queryset(), name)(*args, **kwargs) File "/home/guffi/Documents/projects/whole_genome_ingestion/machado_main/env_mogambo/lib/python3.8/site-packages /Django-3.2.10-py3.8.egg/django/db/models/query.py", line 435, in get raise self.model.DoesNotExist( machado.models.DoesNotExist: Cvterm matching query does not exist.)

I tried to do migrations but it shows

return executor(sql, params, many, context) File "/home/guffi/Documents/projects/whole_genome_ingestion/machado_main/env_mogambo/lib/python3.8/site-packages/Django- 3.2.10-py3.8.egg/django/db/backends/utils.py", line 84, in _execute return self.cursor.execute(sql, params) File "/home/guffi/Documents/projects/whole_genome_ingestion/machado_main/env_mogambo/lib/python3.8/site-packages/Django-3.2.10-py3.8.egg/django/db/utils.py", line 90, in exit raise dj_exc_value.with_traceback(traceback) from exc_value File "/home/guffi/Documents/projects/whole_genome_ingestion/machado_main/env_mogambo/lib/python3.8/site-packages /Django-3.2.10-py3.8.egg/django/db/backends/utils.py", line 82, in _execute return self.cursor.execute(sql) django.db.utils.ProgrammingError: relation "acquisition" already exists

Please help to solve this issue

— Reply to this email directly, view it on GitHub https://github.com/lmb-embrapa/machado/issues/333, or unsubscribe https://github.com/notifications/unsubscribe-auth/ABKNMER6YSLOYVBTPXFFQOTUU2FNLANCNFSM5LOEL7IQ . Triage notifications on the go with GitHub Mobile for iOS https://apps.apple.com/app/apple-store/id1477376905?ct=notification-email&mt=8&pt=524675 or Android https://play.google.com/store/apps/details?id=com.github.android&referrer=utm_campaign%3Dnotification-email%26utm_medium%3Demail%26utm_source%3Dgithub.

You are receiving this because you are subscribed to this thread.Message ID: @.***>

guff95 commented 2 years ago

Thank you for the prompt response the issue that I faced was solved however, I have got one more issue. I loaded all the files Ontologies, Taxonomy, Organism into the database and also loaded new data however while connecting haystack with elastic search. The connection is successful but the rebuild index is not able to load data from database to the elasticsearch. python manage.py rebuild_index Indexing 0 features Rebuild index is not populating the elastic search @azneto

azneto commented 2 years ago

The index contains only features. Have you already loaded GFF files?

Em sex., 14 de jan. de 2022 às 07:47, guff95 @.***> escreveu:

Thank you for the prompt response the issue that I faced was solved however, I have got one more issue. I loaded all the files Ontologies, Taxonomy, Organism into the database and also loaded new data however while connecting haystack with elastic search. The connection is successful but the rebuild index is not able to load data from database to the elasticsearch. python manage.py rebuild_index Indexing 0 features Rebuild index is not populating the elastic search @azneto https://github.com/azneto

— Reply to this email directly, view it on GitHub https://github.com/lmb-embrapa/machado/issues/333#issuecomment-1013010860, or unsubscribe https://github.com/notifications/unsubscribe-auth/ABKNMEVTPBYBPMZTLHVJFY3UV75MTANCNFSM5LOEL7IQ . Triage notifications on the go with GitHub Mobile for iOS https://apps.apple.com/app/apple-store/id1477376905?ct=notification-email&mt=8&pt=524675 or Android https://play.google.com/store/apps/details?id=com.github.android&referrer=utm_campaign%3Dnotification-email%26utm_medium%3Demail%26utm_source%3Dgithub.

You are receiving this because you were mentioned.Message ID: @.***>

syedgufran95 commented 2 years ago

no I have just added above three things and sample fasta files present in the project.

azneto commented 2 years ago

OK. After loading some features, the rebuild_index will index them.

Em sex., 14 de jan. de 2022 às 10:03, syedgufran95 @.***> escreveu:

no I have just added above three things and sample fasta files present in the project.

— Reply to this email directly, view it on GitHub https://github.com/lmb-embrapa/machado/issues/333#issuecomment-1013098838, or unsubscribe https://github.com/notifications/unsubscribe-auth/ABKNMEVYZHL66XVC5F5GGA3UWANJBANCNFSM5LOEL7IQ . Triage notifications on the go with GitHub Mobile for iOS https://apps.apple.com/app/apple-store/id1477376905?ct=notification-email&mt=8&pt=524675 or Android https://play.google.com/store/apps/details?id=com.github.android&referrer=utm_campaign%3Dnotification-email%26utm_medium%3Demail%26utm_source%3Dgithub.

You are receiving this because you were mentioned.Message ID: @.***>

syedgufran95 commented 2 years ago

@azneto I am new to the field of field of bio informatics and doing poc on various tools we can use for wgs ingestion . Can you share a sample feature file(. tb) which I can load??? I couldn't find it on the documentation. Thanks

azneto commented 2 years ago

The 'extras' folder contains a few sample files: https://github.com/lmb-embrapa/machado/tree/master/extras

Em sex., 14 de jan. de 2022 às 13:31, syedgufran95 @.***> escreveu:

@azneto https://github.com/azneto I am new to the field of field of bio informatics and doing poc on various tools we can use for wgs ingestion . Can you share a sample feature file(. tb) which I can load??? I couldn't find it on the documentation. Thanks

— Reply to this email directly, view it on GitHub https://github.com/lmb-embrapa/machado/issues/333#issuecomment-1013274361, or unsubscribe https://github.com/notifications/unsubscribe-auth/ABKNMEQC6RPGIY6O5OL33WDUWBFVLANCNFSM5LOEL7IQ . Triage notifications on the go with GitHub Mobile for iOS https://apps.apple.com/app/apple-store/id1477376905?ct=notification-email&mt=8&pt=524675 or Android https://play.google.com/store/apps/details?id=com.github.android&referrer=utm_campaign%3Dnotification-email%26utm_medium%3Demail%26utm_source%3Dgithub.

You are receiving this because you were mentioned.Message ID: @.***>

syedgufran95 commented 2 years ago

this zip doesn't has the feature file(.tb) @azneto

azneto commented 2 years ago

The feature files are the GFF3 ones. You should observe that it's required to have a matching FASTA file loaded before the GFF3.

Em sex., 14 de jan. de 2022 às 15:52, syedgufran95 @.***> escreveu:

this zip doesn't has the feature file(.tb) @azneto https://github.com/azneto

— Reply to this email directly, view it on GitHub https://github.com/lmb-embrapa/machado/issues/333#issuecomment-1013373592, or unsubscribe https://github.com/notifications/unsubscribe-auth/ABKNMEV4HAMQHREIZB2WVP3UWBWFDANCNFSM5LOEL7IQ . Triage notifications on the go with GitHub Mobile for iOS https://apps.apple.com/app/apple-store/id1477376905?ct=notification-email&mt=8&pt=524675 or Android https://play.google.com/store/apps/details?id=com.github.android&referrer=utm_campaign%3Dnotification-email%26utm_medium%3Demail%26utm_source%3Dgithub.

You are receiving this because you were mentioned.Message ID: @.***>

guff95 commented 2 years ago

Great !!! everything is loaded fine. I am not able to view the data through the jbrowse viewer although I am able to get data through jbrowse API. @azneto

ayazwani commented 2 years ago

Yes I'm also facing the same issue. JBROWSE is not displaying genomes, its showing something like this:
jb whereas on demo application it shows something like this
jb2

azneto commented 2 years ago

@ayazwani JBrowse only shows the chromosome data of regions up to 1kbp. If you hold CTRL and double click, it will zoom in and eventually show the sequence. You can also click 'Full-screen view' to access better navigation tools.

Take a look: https://www.machado.cnptia.embrapa.br/jbrowse_demo/?data=data%2FZea%20mays&loc=Zmays_2%3A232028854..232029038&tracklist=1&nav=1&overview=1&tracks=ref_seq%2Cgene%2Ctranscripts%2CCDS&highlight=

azneto commented 2 years ago

Inactive for more than 10 days