search

StatCan / aaw

Documentation for the Advanced Analytics Workspace Platform
https://statcan.github.io/aaw/
Other
67 stars 12 forks source link

Feedback and requested changes to new portal/landing pages #269

Closed chuckbelisle closed 3 years ago

chuckbelisle commented 3 years ago

We went over some of the new design for the unified portal our portal designers have put together and we decided to offer some constructive feedback in order to better reflect our offering.

List below our needs and wants about today's discussion of the content located on the dev page. (See in Slack for URL)

chuckbelisle commented 3 years ago
sylus commented 3 years ago

Maybe a link to our slide deck and stratosphere video as well which shows the whole platform?

YouTube: https://www.youtube.com/watch?v=quYuuEAqNm0 SlideDeck: https://t.co/1D3bti0WTm?amp=1

sylus commented 3 years ago

@chuckbelisle what is the best way to get this done, send them a mock-up?

blairdrummond commented 3 years ago

@sylus I am leaving comments in the Adobe XD doc and will ping that team tomorrow to follow-up.

My current list

Anything else?

sylus commented 3 years ago

Think that is it from me, we are going to do a little write up to help for each point message it back here.

sylus commented 3 years ago

Hey @blairdrummond and @chuckbelisle me and @zachomedia gave a go at this and tried to stay at a high level.

daaas-txt.pdf

(Links in the PDF)

Advanced Analytics Workspace (AAW)

The Advanced Analytics Workspace provides options to explore and analyze your data, and turn your insights into sharable solutions. It is built using Open Source and Open Standard technologies, aligning with the latest Treasury Board Secretariat (TBS) policy on information systems. We provide access to the latest data science technologies in a scalable and self-service environment.

• YouTube: https://www.youtube.com/watch?v=quYuuEAqNm0&feature=youtu.be • Slide deck: https://govcloud.blob.core.windows.net/docs/daaas-cncf.pdf

Analysis

// Data analysis, data management, machine learning

Jupyter Notebooks

Use Jupyter Notebooks, RStudio and Linux desktops for analysis and machine learning development.

Pipelines

Run flexible and scalable data pipelines powered by Argo workflows. See our advanced MLOps pipeline for inspiration.

Object Storage (MinIO)

Powered by MinIO, you can store private and shared files in a centralized object storage environment. Browse, upload/download and share using a web interface or a command line. It is fully S3 API compatible, so it works well with pipelines and other programs. It's also automounted into your Jupyter Notebooks for easy access.

MLFlow

Track your machine learning models through your individual MlFlow central artifact repository.

Advanced Data Storage

Full integration thanks to Goofys with other data storage systems, such as Azure Blob Storage, Azure Data Lakes, and other S3 compatible API's

Platform as a Service (PaaS) Integrations

Use available PaaS services to enhance the tools provided to you by the AAW. Use cloud-provided storage, compute and analysis tools right from your Jupyter Notebooks and Kubeflow Pipelines. Examples include Azure Databricks (Spark), Azure Machine Learning (ML), Azure Data Lakes and more!

Dashboard and API's

// Visualize, publish, host

Shiny Server

Publish interactive dashboards to allow visitors to interact with your data and your analysis.

Seldon Core

Serve your machine learning models via a REST API.

Advanced

// Security and Observability

Secrets generation and management

Powered by Hashicorp Vault, we securely generate and manage credentials for use with various systems. It is used under the hood to provide MinIO credentials to Jupyter Notebooks, Boathouse and other services.

Service Mesh

Powered by Istio, in-cluster services are protected via mutual TLS authentication.

Serverless

Today we offer the Serving component of the Knative serverless platform. In the near future, we will expand this integration to the whole Knative serverless platform.

Visualize and Discover Data

We offer individual ElasticSearch and Kibana instances for your use.

Open Source Contributions

Boathouse

Simplify access to your object storage by making it available in your Jupyter Notebooks.

Jupyter API's

Rewritten backend in Golang featuring numerous user interface improvements and integration with KubeCost.

See https://github.com/StatCan for more open source tools.

Made with ❤ by the Statistics Canada Cloud Native team and powered by Kubernetes. Learn more.

sylus commented 3 years ago

Everything looks good @blairdrummond?

blairdrummond commented 3 years ago

Yeah looks good to me - few things

I won't make promises on the inclusion of Service Mesh (unless we start using OPA Istio Plugin or something), since it can't really be used by the users. But I might be able to bundle it into a larger platform comment about how Kubernetes offers X,Y,Z that makes things more secure, maintainable, observable, etc?

I might get pushback on explicitly naming the TBS policy too, but I'm going to push for the link as I think that can accomplish the point.

You're thinking the 

See https://github.com/StatCan for more open source tools.

Made with heart by the Statistics Canada Cloud Native team and powered by
Kubernetes. Learn more.

as a page footer?

Also I might pop out the bit about Argo workloads but I have some copy for Pipelines already that didn't get included yet, is that content good?

I'll push for these ^ we'll also need to quickly throw placeholders in the wiki for

CC @chuckbelisle

sylus commented 3 years ago

I think for the Service Mesh it was that we give access to Kiali so people can see their traces live, and that they instantly get mutual tls so security is "baked-in" which we wanted to highlight.

Is fine to pop out the argo workloads if you have a good paragraph for that.

And yeah the other part would be for the footer. (reminder we linked to our platform docs for that sentence, see the PDF for links. As we wanted to show all of the platform stuff users inherit.

@blairdrummond @zachomedia

blairdrummond commented 3 years ago

Ooh if we have Kiali available for users then for sure, I might add it as "Kiali" then? I wanted to add Grafana and Kubecost as well, and then we have a comprehensive monitoring section.

sylus commented 3 years ago

I think just leave it as service mesh but can add a description there keeping it high level, not sure how many sections they would let us add and we tried to keep it simple.

Will leave up to you, just want them to action most of what we wrote.

chuckbelisle commented 3 years ago

The dev page for the new portal has been updated with a lot of our requests and recommendations. Please have a look https://daaas-dev.covid.cloud.statcan.ca/

chuckbelisle commented 3 years ago

I think this ticket can be closed @blairdrummond . Please confirm

chuckbelisle commented 3 years ago

Content has been verified and passed UAT.