Prepare AI workshop StreamLit app & Ersilia models on AWS

[miquelduranfrigola]

Hi @sucksido,

These are tasks that are absolutely important to complete before the workshop this week. As discussed over our brief catch up today, here is a breakdown of what needs to be done to make sure the workshop goes smoothly.

• [x] We need to provide the Streamlit app through a URL and not an IP address, so a DNS record needs to be set up for this. Note that this is primarily required for the Streamlit app for the workshop, and not individual models.
• [x] The Streamlit app needs to be deployed on AWS, potentially using AWS Fargate. The necessary Dockerfile is within the app's repository.
• [x] To ensure availability, we need two deployment environments for each of a) the Streamlit app, b) models eos43at and eos9ei3. One environment will be used solely by Ersilia to demo during the workshop, while the others will be made available to users.
• [x] Stress testing of the StreamLit app across multiple simultaneous sessions.

Timeline: Workshop is on Friday the 24th of May at 8:00am. Ideally, we would have this deployment ready by Wednesday. This is priority over ZairaChem deployment on AWS.

[sucksido]

Hi @miquelduranfrigola and @DhanshreeA , Thanks for adding the notes, I will update the issue as I progress

[sucksido]

Hi @miquelduranfrigola and @GemmaTuron , Need the following access to be able to create DNS records: { "Version": "2012-10-17", "Statement": [ { "Effect": "Allow", "Action": [ "route53:ListHostedZones", "route53:ChangeResourceRecordSets", "route53:GetChange", "route53:ListResourceRecordSets" ], "Resource": "*" } ] }

[sucksido]

Hi @miquelduranfrigola and @miquelduranfrigola, I need the following access for me to able able to see my security credentials.: { "Version": "2012-10-17", "Statement": [ { "Effect": "Allow", "Action": [ "iam:GetUser", "iam:ListAccessKeys", "iam:CreateAccessKey", "iam:DeleteAccessKey", "iam:GetAccessKeyLastUsed" ], "Resource": "arn:aws:iam::*:user/${aws:username}" } ] }

[sucksido]

I am busy setting AWS locally and need the credentials for that: AWS Access Key ID [None]: YOUR_ACCESS_KEY_ID AWS Secret Access Key [None]: YOUR_SECRET_ACCESS_KEY Default region name [None]: eu-central-1 Default output format [None]: json

[DhanshreeA]

Hey @sucksido let me set these up for you.

[sucksido]

Thanks @DhanshreeA , Please let me know once it's sorted

[DhanshreeA]

Hi @sucksido all done, let me know if it works.

[DhanshreeA]

I am busy setting AWS locally and need the credentials for that: AWS Access Key ID [None]: YOUR_ACCESS_KEY_ID AWS Secret Access Key [None]: YOUR_SECRET_ACCESS_KEY Default region name [None]: eu-central-1 Default output format [None]: json

I understand that I don't need to do anything here with the IAM Permissions policy that I have created for your user, right?

[sucksido]

@DhanshreeA Awesome it worked now, thanks

[GemmaTuron]

@sucksido

The app is not working from the latest link you provide, this is what I get when I try to do Train a Model

AttributeError: module 'umap' has no attribute 'UMAP'
Traceback:
File "/usr/local/lib/python3.10/site-packages/streamlit/runtime/scriptrunner/script_runner.py", line 600, in _run_script
    exec(code, module.__dict__)
File "/app/app.py", line 149, in <module>
    fig3 = do_plot_umap(X, y)
File "/usr/local/lib/python3.10/site-packages/streamlit/runtime/caching/cache_utils.py", line 165, in wrapper
    return cached_func(*args, **kwargs)
File "/usr/local/lib/python3.10/site-packages/streamlit/runtime/caching/cache_utils.py", line 196, in __call__
    return self._get_or_create_cached_value(args, kwargs)
File "/usr/local/lib/python3.10/site-packages/streamlit/runtime/caching/cache_utils.py", line 221, in _get_or_create_cached_value
    return self._handle_cache_miss(cache, value_key, func_args, func_kwargs)
File "/usr/local/lib/python3.10/site-packages/streamlit/runtime/caching/cache_utils.py", line 277, in _handle_cache_miss
    computed_value = self._info.func(*func_args, **func_kwargs)
File "/app/app.py", line 88, in do_plot_umap
    return plot_umap(X, y)
File "/app/plots.py", line 60, in plot_umap
    reducer = umap.UMAP(n_components=2)

[GemmaTuron]

I think the latest commit to the repo fixes this. @sucksido how does the online deployment get updated? Can it get updated everytime there is a push in github?

[sucksido]

Hi @GemmaTuron i tried to pull the latest changes and run the app locally and re-deployed it to the AWS env, I am still getting the same error when I try to train a model. Currently the Docker container is being deployed manually, to automate this we will need to add a workflow on github to pick changes when they are made and run the deployment

[GemmaTuron]

Do you have a way to check if umap is properly installed? I tried it out in digital ocean and it worked the package has to be umap-learn

[sucksido]

Yes I can try locally and advise if it work, please gimme a few mins

[sucksido]

Hi @miquelduranfrigola @DhanshreeA @GemmaTuron UPDATE: all I need now is a domain to be registered to be able to add the DNS entries for Streamlit and the indvidual models, stress testing will be be completed today, I will share the report once executed

[DhanshreeA]

Hi @sucksido do you need any help with that?

[sucksido]

Hi @DhanshreeA , I just need confirmation interms of the domain name and if we are going for .io, .com or .net then I can create it and map it to the public IPs

[sucksido]

Update: the only thing outstanding now is to create DNS entries for individual models and streamlit

[GemmaTuron]

Let's make this app dormant for the moment as we continue working on the platform

[GemmaTuron]

@sucksido feel free to mark it as done and close the issue once the app is dormant/destroyed

[sucksido]

@GemmaTuron done