search

finos / community

FINOS Community, Project and SIG wide collaboration space
http://community.finos.org
66 stars 28 forks source link

Special Interest Group - AI Readiness #303

Closed ColinEberhardt closed 3 weeks ago

ColinEberhardt commented 10 months ago

Business Problem

The recent advances in AI (and more specifically Generative AI) are poised to have a disruptive impact across both our business and home lives. The rapid advances (that are showing no sign of slowing) are leading to a wealth or potential that touches upon the products we offer to clients, our personal productivity as well as the operational aspects of our business. Very few technologies in the past have had such a broad 'reach'.

If you need any further persuading, McKinsey predict an impact of circa $200 billion on the Banking Industry.

There are certainly challenges and issues with the current technology (hallucinations, prompt injections), but thing are moving at pace, and the technology challenges we see today may not exist tomorrow.

There is a very strong desire to on-board, trial, experiment and release products using this technology across the FINOS membership and broader FS community. However, as an industry we have some unique challenges that the vendors, academics, and startups are not going to solve for us.

All banks have mature processes for on-boarding technology, however, (Generative) AI presents some new challenges that the existing processes may not be well-suited to addressing. Without going into the details, there is much work needed to allow the 'safe' use of AI (where safety considers both the customer and the bank) and ultimately allow FS organisations to rapidly adopt new technologies as they emerge.

Proposed Solution

The goal of this SIG is to provide a space to discuss challenges that are common to all members. This has parallels with Open Source Readiness and the SIG may choose to follow similar initiatives, creating educational materials, training and raising awareness. However, there is the opportunity to go further and mutually develop a governance framework that managed the on-boarding, development of, and running AI-based solutions within FS organisation. Ideally this would allow us to get ahead of future regulations.

Current State

There have been numerous conversations with organisations who have expressed an interest and nominated individuals who would participate in the SIG. Currently we have commitment from:

We also have a growing list of organisation who have expressed and interest, and we expect commitment to follow shortly.

Existing Materials

None relevant

SIG Maintainers

Colin Eberhardt (@ColinEberhardt), Scott Logic CTO, FINOS TOC chair has volunteered to co-chair this group. We have an expression of interest from another co-chair, with details following shortly.

SIG Contribution and Onboarding process (v. 1.0, last updated on October 27, 2020)

Below is the list of tasks that FINOS Team and the contribution author goes through in order to complete the FINOS SIG onboarding process. Please do not edit these contents at contribution time!

Voting (SIG Lead and FINOS ED)

Identify project meta (Lead: FINOS SIG POC, Support: FINOS Marketing)

Project Communication Channel(s)

Repository (Lead: FINOS Infra)

Infra setup (Lead: FINOS Infra)

Metadata update (Lead: FINOS Infra)

Update FINOS SIG Roster

Announcement (Lead: FINOS POC)

Press Release (OPTIONAL) (Lead: FINOS POC)

Onboarding and training

Contributors Metadata (Lead: FINOS SIG POC, Support: FINOS infra)

MikeWilsonZngly commented 10 months ago

@ColinEberhardt. This is a great idea and we would be delighted to take part.We've deployed across ZNGLY conversational AI using Google Vertex and PaLM. This is across all media types - PDFs, videos, sharpoint sources too. It's being used in production today with a range of fintech and insurtech clients, and the banks they are working with. Delighted to share our experience with the group - it was a significant undertaking - and great to share what we learnt.

Neetuj commented 10 months ago

I love the Idea. I learned a lot from OSR and helped there a bit too. Will love to learn and contribute to this SIG too

BrittanyIstenes commented 10 months ago

@ColinEberhardt - I would like to sit down and discuss the FNMA participation within this SIG. Currently I sit on the co-chair for OSR and InnerSource. I would need to more than likely drop one to participate.

tenatzen commented 10 months ago

👍🏼 You've got the backing of all of us at the Zenith SIG on this one

caradelia commented 10 months ago

@ColinEberhardt this is a great idea for a SIG. With seeing full support from the Zenith SIG, I see no issues.

awaiken commented 10 months ago

definitely good idea and I'm sure I can get someone from my firm to be active here

opatrascoiu commented 10 months ago

this is a good idea 👍

cm-howard commented 10 months ago

Very supportive of this and I'd be very keen to get EPAM engaged and support bringing along some new individuals to contribute too.

pcheong-lbg commented 10 months ago

Great idea

cronky commented 10 months ago

Very keen to get involved. As some may know I've written quite a bit on the Enterprise Readiness topic (i.e. https://blog.scottlogic.com/2023/05/04/generative-ai-solution-architecture.html and this has also come up on Architect Tomorrow a few times - i.e. https://www.youtube.com/watch?v=l-zHfUkVYzc) and have experience with architecture for deploying and scaling advanced ML architectures for Financial services: https://www.youtube.com/watch?v=o3OF5aRSQco&

pholleran commented 10 months ago

I and GitHub are interested in participating 😄

ajgnacht commented 10 months ago

I and Microsoft are supportive and interested in participating.

ColinEberhardt commented 10 months ago

Thanks for all the support so for. Interestingly the UK government have just published a framework for GenAI:

https://www.gov.uk/government/publications/generative-ai-framework-for-hmg/generative-ai-framework-for-hmg-html

But this document is an ‘advice’ framework, not a ‘governance’ framework, which those trying to use this technology very much need.

mcleo-d commented 10 months ago

👋🏻 - It would be great to have a round table to discuss the purpose & governance surrounding the SIG as this could be a prime opportunity to gather requirements from banks without contributing materials back to an open standard / project.

Also, it would be great to have a FINOS member bank(s) as part of the leadership to support the nomination, help set the direction / agenda and keep the SIG delivering openly for the benefit of the industry.

This is extremely positive, but I do think we need to understand the community rules of engagement before fully committing firms and people.

brunodom commented 10 months ago

@ColinEberhardt I am supporting this initiative on behalf of Intel and have active participation in the SIG

vicenteherrera commented 9 months ago

I'm already participating on the Common Cloud Controls, and would like also to participate on this one.

jstclair2019 commented 9 months ago

@ColinEberhardt would welcome participation here. I am working with a state non-profit coding academy and we have a grant for statewide AI readiness and adoption.

ferguskeenan commented 9 months ago

Keen to be involved as my company has a lot of experience managing more traditional software adoption within FS and it would be interesting to contribute that knowledge to this new problem space.

TheFoxAtWork commented 9 months ago

👋🏻 I would love to be a participant or support a leadership role depending on expected time commitment and availability. I'm currently keeping track of several AI and AI security related efforts across two foundations (among many other tasks and activities)

eddie-knight commented 9 months ago

Myself and others from Sonatype are interested in supporting this effort.

miglaros commented 9 months ago

From an onboarding perspective one of the things that stands out to me is the relative distance from the GenAI incubation work that tends to take place in most banks innovation centres and the operational risk assessments needing to be conducted by Information Security. All too often Innovation is understandably early without the benefit of fully formed learnings and engineering feedback whilst Info Sec is harried with yet another emerging tech to ramp up on.

I think there exists a happy middle with a light weight self-assessment process inspired by Microsoft's RAI standard (https://blogs.microsoft.com/wp-content/uploads/prod/sites/5/2022/06/[Microsoft-Responsible-AI-Standard-v2-General-Requirements-3.pdf](https://blogs.microsoft.com/wp-content/uploads/prod/sites/5/2022/06/Microsoft-Responsible-AI-Standard-v2-General-Requirements-3.pdf)) and OWASP’s Top Ten for LLM applications https://owasp.org/www-project-top-10-for-large-language-model-applications/.

Perhaps in the GitHub we could contribute some potential assessment checklist ideas/approaches to be completed as part of onboarding best practice?

rocketstack-matt commented 9 months ago

There is crossover interest from us in the DevOps Automation SIG, @ashukla13 has raised the topic of use specifically within the context of AI Code Assistants (https://github.com/finos/devops-automation/issues/179) and was planning a session dedicated to this topic.

One thing I think would be useful to come out of such a framework would be industry agreed metrics for measuring the effectiveness of GenAI in development. This report from GitClear highlights potential issues from increased code churn (code committed to the repo and subsequently reverted or changed within two weeks).

Whilst we thankfully moved away from counting lines of code as a measure of productivity long ago, we need to ensure we don't inadvertently bring it back as some measure of success for AI code assistants.

ColinEberhardt commented 9 months ago

From an onboarding perspective one of the things that stands out to me is the relative distance from the GenAI incubation work that tends to take place in most banks innovation centres and the operational risk assessments needing to be conducted by Information Security. All too often Innovation is understandably early without the benefit of fully formed learnings and engineering feedback whilst Info Sec is harried with yet another emerging tech to ramp up on.

Hi @miglaros - completely agree, and this is the issue we want to tackle through this group.

ashukla13 commented 9 months ago

The topic that the DevOps SIG was interested in discussing at the next meeting is https://github.com/finos/devops-automation/issues/181 This is a narrow subset of the broader set of issues around AI Readiness

vicenteherrera commented 9 months ago

I'm very interested in collaborating with this, as we at Control Plane are right now doing research on AI security.

miglaros commented 9 months ago

Perhaps we can draw inspiration from this pattern and establish some sample code/demo application/reference implementation: https://mlops.community/how-to-build-a-knowledge-assistant-at-scale/

mindthegab commented 9 months ago

Thank you @ColinEberhardt and everyone involved in this proposal. I am happy to report that the group was successfully unanimously approved by the FINOS Governing Board on Monday - so we're off to the races! Thank you for the overwhelming interest and support.

The group was approved to start as FINOS Member only for its initial formation phase (3/6 months) and then go fully open, so if you are a FINOS member please do reach out to @ColinEberhardt or to membersuccess@finos.org to be added to the meetings and activities. If you are not a member and you'd like to join as FINOS member you can do so here or reach out to membership@finos.org.

We will be announcing this publicly, as part of the broader AI Strategic Initiative, in a couple of weeks, so appreciate coordinating with marketing@finos.org for further public promotion of this activity.

Thanks again for your support and looking forward to another amazingly impactful collaboration here!

satyampsoni commented 8 months ago

How to get involved ?

ColinEberhardt commented 8 months ago

Hi @satyampsoni you can find further information here: https://github.com/finos/ai-readiness

salmenus commented 7 months ago

Hi everyone. I worked on trading tech at Goldman, BofA, and other banks, and contributed to AG Grid. I recently started an OS project to simplify the integration of Gen AI into web apps (fin use-case here).

I've been thinking about some of the topics raised here — issues like output filtering, supporting explainability, cross-applications Gen AI integration, etc; I'll be happy to contribute to the conversation.

Thanks @mcleo-d for pointing me to this conversation and this SIG.

ColinEberhardt commented 3 weeks ago

This group has now been formed, see https://github.com/finos/ai-readiness