ivmfnal / data_dispatcher

BSD 3-Clause "New" or "Revised" License
1 stars 3 forks source link

Strange error starting today - may be me but I can't figure it out #4

Closed hschellman closed 1 year ago

hschellman commented 1 year ago

I'm having trouble with the create project command - I found it while debugging python but now it fails on straight command line.

This is using Jake's virtual environment which I don't believe has changed recently. Is this a DB issue or user error. It claims it can't insert into files.

(venv) dd project create 'files from schellma:run5141recentReco where namespace="pdsp_det_reco" ordered skip 10 limit 100'

Traceback (most recent call last): File "/dune/app/users/calcuttj/dd_metacat_pip/venv/bin/dd", line 8, in sys.exit(main()) File "/dune/app/users/calcuttj/dd_metacat_pip/venv/lib/python3.9/site-packages/data_dispatcher/ui/ui_main.py", line 155, in main cli.run(sys.argv, argv0="dd") File "/dune/app/users/calcuttj/dd_metacat_pip/venv/lib/python3.9/site-packages/data_dispatcher/ui/cli/cli.py", line 205, in run self._run(command, context, argv, usage_on_error) File "/dune/app/users/calcuttj/dd_metacat_pip/venv/lib/python3.9/site-packages/data_dispatcher/ui/cli/cli.py", line 200, in _run return interp._run(pre_command + word, context, rest, usage_on_error = usage_on_error) File "/dune/app/users/calcuttj/dd_metacat_pip/venv/lib/python3.9/site-packages/data_dispatcher/ui/cli/cli.py", line 200, in _run return interp._run(pre_command + word, context, rest, usage_on_error = usage_on_error) File "/dune/app/users/calcuttj/dd_metacat_pip/venv/lib/python3.9/site-packages/data_dispatcher/ui/cli/cli.py", line 99, in _run return self(command, context, opts, args) File "/dune/app/users/calcuttj/dd_metacat_pip/venv/lib/python3.9/site-packages/data_dispatcher/ui/ui_project.py", line 112, in call info = client.create_project(files, common_attributes=common_attrs, project_attributes=project_attrs, query=query, worker_timeout=worker_timeout) File "/dune/app/users/calcuttj/dd_metacat_pip/venv/lib/python3.9/site-packages/data_dispatcher/api.py", line 244, in create_project return self.post("create_project", json.dumps( File "/dune/app/users/calcuttj/dd_metacat_pip/venv/lib/python3.9/site-packages/data_dispatcher/api.py", line 126, in post raise APIError(url, response.status_code, response.text) data_dispatcher.api.APIError: DataDispatcherServer error: URL: https://metacat.fnal.gov:9443/dune/dd/data/create_project HTTP status code: 500 Message: Message from the server:

Application error

relation "files" does not exist LINE 2: insert into files(namespace, name) ^

Traceback (most recent call last):
  File "/home/dbidata3/multiserver/webpie/webpie/WPApp.py", line 715, in wsgi_call
    response = root_handler._handle_request(request, "", path_down, args)
  File "/home/dbidata3/multiserver/webpie/webpie/WPApp.py", line 483, in _handle_request
    return subhandler(request, relpath, **args)
  File "", line 75, in create_project
  File "./data_dispatcher/data_dispatcher/db.py", line 580, in add_files
    DBFile.create_many(self.DB, files_descs)
  File "./data_dispatcher/data_dispatcher/db.py", line 802, in create_many
    c.execute(f"""
psycopg2.errors.UndefinedTable: relation "files" does not exist
LINE 2:                 insert into files(namespace, name)
                                    ^

hschellman commented 1 year ago

That lost a bit at the end:

(venv) dd project create 'files from schellma:run5141recentReco where namespace="pdsp_det_reco" ordered skip 10 limit 100' Traceback (most recent call last): File "/dune/app/users/calcuttj/dd_metacat_pip/venv/bin/dd", line 8, in sys.exit(main()) File "/dune/app/users/calcuttj/dd_metacat_pip/venv/lib/python3.9/site-packages/data_dispatcher/ui/ui_main.py", line 155, in main cli.run(sys.argv, argv0="dd") File "/dune/app/users/calcuttj/dd_metacat_pip/venv/lib/python3.9/site-packages/data_dispatcher/ui/cli/cli.py", line 205, in run self._run(command, context, argv, usage_on_error) File "/dune/app/users/calcuttj/dd_metacat_pip/venv/lib/python3.9/site-packages/data_dispatcher/ui/cli/cli.py", line 200, in _run return interp._run(pre_command + word, context, rest, usage_on_error = usage_on_error) File "/dune/app/users/calcuttj/dd_metacat_pip/venv/lib/python3.9/site-packages/data_dispatcher/ui/cli/cli.py", line 200, in _run return interp._run(pre_command + word, context, rest, usage_on_error = usage_on_error) File "/dune/app/users/calcuttj/dd_metacat_pip/venv/lib/python3.9/site-packages/data_dispatcher/ui/cli/cli.py", line 99, in _run return self(command, context, opts, args) File "/dune/app/users/calcuttj/dd_metacat_pip/venv/lib/python3.9/site-packages/data_dispatcher/ui/ui_project.py", line 112, in call info = client.create_project(files, common_attributes=common_attrs, project_attributes=project_attrs, query=query, worker_timeout=worker_timeout) File "/dune/app/users/calcuttj/dd_metacat_pip/venv/lib/python3.9/site-packages/data_dispatcher/api.py", line 244, in create_project return self.post("create_project", json.dumps( File "/dune/app/users/calcuttj/dd_metacat_pip/venv/lib/python3.9/site-packages/data_dispatcher/api.py", line 126, in post raise APIError(url, response.status_code, response.text) data_dispatcher.api.APIError: DataDispatcherServer error: URL: https://metacat.fnal.gov:9443/dune/dd/data/create_project HTTP status code: 500 Message: Message from the server:

Application error

relation "files" does not exist LINE 2: insert into files(namespace, name) ^

Traceback (most recent call last):
  File "/home/dbidata3/multiserver/webpie/webpie/WPApp.py", line 715, in wsgi_call
    response = root_handler._handle_request(request, "", path_down, args)
  File "/home/dbidata3/multiserver/webpie/webpie/WPApp.py", line 483, in _handle_request
    return subhandler(request, relpath, **args)
  File "", line 75, in create_project
  File "./data_dispatcher/data_dispatcher/db.py", line 580, in add_files
    DBFile.create_many(self.DB, files_descs)
  File "./data_dispatcher/data_dispatcher/db.py", line 802, in create_many
    c.execute(f"""
psycopg2.errors.UndefinedTable: relation "files" does not exist
LINE 2:                 insert into files(namespace, name)
                                    ^

hschellman commented 1 year ago

Looks like GitHub messes with the formatting on the dump. Probably just need to try the command yourself.

ivmfnal commented 1 year ago

This was the error I was talking about in Slack when I said that DD may not work correctly while I am upgrading it to add the idle timeout feature