hysds / pele

REST API for HySDS Datasets
Apache License 2.0
0 stars 0 forks source link

hot-fix: Pin Jinja2 to 3.0.3 #22

Closed mcayanan closed 2 years ago

mcayanan commented 2 years ago

This hot fix pins the Jinja2 version to 3.0.3 due to cluster provisioning failing with the following error when it pulled the latest, which was version 3.1.0:

module.common.aws_instance.mozart (remote-exec): [100.104.3.67] run: flask create-db
module.common.aws_instance.mozart (remote-exec): [100.104.3.67] out: Traceback (most recent call last):
module.common.aws_instance.mozart (remote-exec): [100.104.3.67] out:   File "/export/home/hysdsops/sciflo/bin/flask", line 8, in <module>
module.common.aws_instance.mozart (remote-exec): [100.104.3.67] out:     sys.exit(main())
module.common.aws_instance.mozart (remote-exec): [100.104.3.67] out:   File "/export/home/hysdsops/sciflo/lib/python3.9/site-packages/flask/cli.py", line 995, in main
module.common.aws_instance.mozart (remote-exec): [100.104.3.67] out:     cli.main(args=sys.argv[1:])
module.common.aws_instance.mozart (remote-exec): [100.104.3.67] out:   File "/export/home/hysdsops/sciflo/lib/python3.9/site-packages/flask/cli.py", line 601, in main
module.common.aws_instance.mozart (remote-exec): [100.104.3.67] out:     return super().main(*args, **kwargs)
module.common.aws_instance.mozart (remote-exec): [100.104.3.67] out:   File "/export/home/hysdsops/sciflo/lib/python3.9/site-packages/click/core.py", line 1053, in main
module.common.aws_instance.mozart (remote-exec): [100.104.3.67] out:     rv = self.invoke(ctx)
module.common.aws_instance.mozart (remote-exec): [100.104.3.67] out:   File "/export/home/hysdsops/sciflo/lib/python3.9/site-packages/click/core.py", line 1653, in invoke
module.common.aws_instance.mozart (remote-exec): [100.104.3.67] out:     cmd_name, cmd, args = self.resolve_command(ctx, args)
module.common.aws_instance.mozart (remote-exec): [100.104.3.67] out:   File "/export/home/hysdsops/sciflo/lib/python3.9/site-packages/click/core.py", line 1700, in resolve_command
module.common.aws_instance.mozart (remote-exec): [100.104.3.67] out:     cmd = self.get_command(ctx, cmd_name)
module.common.aws_instance.mozart (remote-exec): [100.104.3.67] out:   File "/export/home/hysdsops/sciflo/lib/python3.9/site-packages/flask/cli.py", line 557, in get_command
module.common.aws_instance.mozart (remote-exec): [100.104.3.67] out:     return info.load_app().cli.get_command(ctx, name)
module.common.aws_instance.mozart (remote-exec): [100.104.3.67] out:   File "/export/home/hysdsops/sciflo/lib/python3.9/site-packages/flask/cli.py", line 411, in load_app
module.common.aws_instance.mozart (remote-exec): [100.104.3.67] out:     app = locate_app(self, import_name, None, raise_if_not_found=False)
module.common.aws_instance.mozart (remote-exec): [100.104.3.67] out:   File "/export/home/hysdsops/sciflo/lib/python3.9/site-packages/flask/cli.py", line 260, in locate_app
module.common.aws_instance.mozart (remote-exec): [100.104.3.67] out:     __import__(module_name)
module.common.aws_instance.mozart (remote-exec): [100.104.3.67] out:   File "/export/home/hysdsops/sciflo/ops/pele/app.py", line 6, in <module>
module.common.aws_instance.mozart (remote-exec): [100.104.3.67] out:     from pele import create_app, db
module.common.aws_instance.mozart (remote-exec): [100.104.3.67] out:   File "/export/home/hysdsops/sciflo/ops/pele/pele/__init__.py", line 11, in <module>
module.common.aws_instance.mozart (remote-exec): [100.104.3.67] out:     from pele.extensions import (cache, assets_env, debug_toolbar, login_manager, cors, bcrypt, db, limiter, mail)
module.common.aws_instance.mozart (remote-exec): [100.104.3.67] out:   File "/export/home/hysdsops/sciflo/ops/pele/pele/extensions.py", line 21, in <module>
module.common.aws_instance.mozart (remote-exec): [100.104.3.67] out:     debug_toolbar = DebugToolbarExtension()
module.common.aws_instance.mozart (remote-exec): [100.104.3.67] out:   File "/export/home/hysdsops/sciflo/lib/python3.9/site-packages/flask_debugtoolbar/__init__.py", line 50, in __init__
module.common.aws_instance.mozart (remote-exec): [100.104.3.67] out:     self.jinja_env = Environment(
module.common.aws_instance.mozart (remote-exec): [100.104.3.67] out:   File "/export/home/hysdsops/sciflo/lib/python3.9/site-packages/jinja2/environment.py", line 363, in __init__
module.common.aws_instance.mozart (remote-exec): [100.104.3.67] out:     self.extensions = load_extensions(self, extensions)
module.common.aws_instance.mozart (remote-exec): [100.104.3.67] out:   File "/export/home/hysdsops/sciflo/lib/python3.9/site-packages/jinja2/environment.py", line 117, in load_extensions
module.common.aws_instance.mozart (remote-exec): [100.104.3.67] out:     extension = t.cast(t.Type["Extension"], import_string(extension))
module.common.aws_instance.mozart (remote-exec): [100.104.3.67] out:   File "/export/home/hysdsops/sciflo/lib/python3.9/site-packages/jinja2/utils.py", line 149, in import_string
module.common.aws_instance.mozart (remote-exec): [100.104.3.67] out:     return getattr(__import__(module, None, None, [obj]), obj)
module.common.aws_instance.mozart (remote-exec): [100.104.3.67] out: AttributeError: module 'jinja2.ext' has no attribute 'with_'

In reading the Jinja 3.1.0 Change Release, https://jinja.palletsprojects.com/en/3.1.x/changes/, it removed the with_ attribute:

https://github.com/pallets/jinja/pull/1544/files#diff-8dbf8b15a7c614464f5bd0c738618e7efd94e1dc45cb2158b6d9b2826fab9ff1L877

and judging from the stacktrace, it appears that changes would need to be made to the Flask DebugToolBar module to resolve:

https://github.com/flask-debugtoolbar/flask-debugtoolbar/blob/master/flask_debugtoolbar/__init__.py#L62

mcayanan commented 2 years ago

@DustinKLo I'm not sure. I had thought we needed to initially, but according to the code, sdscli isn't being installed on GRQ, which is where the error is happening:

https://github.com/sdskit/sdscli/blob/8594cdd4593a0d474ff0c44b403b4b8103c8eca7/sdscli/adapters/hysds/update.py#L417-L441

I should probably go through a force branch to see if it goes through all the way.

DustinKLo commented 2 years ago

good point, feel free to merge 👍

mcayanan commented 2 years ago

Force branch job passed successfully: https://nisar-pcm-ci.jpl.nasa.gov/job/force-branches/job/mkarim-force_branch-E2E-test/125/