search

finos / community

FINOS Community, Project and SIG wide collaboration space
http://community.finos.org
66 stars 28 forks source link

TimeBase Community Edition (WIP) #70

Closed stuartfarr closed 3 years ago

stuartfarr commented 4 years ago

Business Problem

Time-series data is foundational to both buy-side and sell-side financial services firms. The need for time-series data management systems is well understood and is well-served by products provided under commercial licenses. There are also many open-source time-series data systems but their uptake in financial services firms is less extensive, possibly due to insufficient performance and scalability or lack of appropriateness for financial services time-series data. There is constant demand to reduce total cost of operating technology stacks, whilst at the same time implementing changes to serve new opportunities or meet new obligations. Implementing change in a cost-effective manner has led to an increase in the use of cloud and open-source technologies, and less of a “one size fits all” approach.

Proposed Solution

Having been battle-tested in trading firms for over 10 years under a commercial license, TimeBase is now offered under an open-source license as TimeBase Community Edition”. TimeBase Community Edition was designed from inception for managing time-series data found in financial services.

The goal is to provide enterprise grade time-series data management and messaging capability which enables financial services firms to use with other open-source and commercial components to augment existing solutions, create new solutions.

TimeBase can be used, simultaneously, as a traditional time-series database and real-time data messaging/streaming server. Data structures (“streams”) are user-defined. There is a rich API for interacting (ingesting, streaming) with TimeBase, which provide the necessary hooks for data population and data utilization.

Current State

Summarize the history and current state of the project

Existing Materials

If materials already exist, provide a link to them that Foundation staff can access - if it's in a private GitHub.com repositories, you should invite the finos-admin user with R/O permissions to those repositories

Development Team

List out all of the current team members, including full name, affiliation, work email address, and GitHub.com username

Contribution process

Below is the list of tasks that FINOS Team and the contribution author go through in order to complete the FINOS contribution process. Please do not edit these contents at contribution time!

FINOS Contrib POC

Identify project meta (Lead: FINOS Contrib POC, Support: FINOS Marketing)

Maintainers, contributors and CLAs (Lead: FINOS Contrib POC, Support: FINOS infra)

Code validation (only if code is contributed) (Lead: FINOS Infra)

Voting (Lead: FINOS Infra)

Code transfer (Lead: FINOS Infra)

Announcement (Lead: FINOS Contrib POC)

Infra setup (Lead: FINOS Infra)

First Release

Onboarding and training

avavilau commented 4 years ago

Current State: History: 2006 – Timebase 1.0 (historical data analytics, SQL)​ 2010 – Timebase 2.0 (timeseries data base, live streaming)​ 2015 – Timebase 5.0 (Very large databases, HDFS)​ 2018 – IPC/UDP low latency mode (Timebase topics)​ 2019 – Cloud support (REST/WS, Docker, Kuber)​ 2020 – Timebase 6.0 Community Edition (open source) and Clustering

TimeBase 6.0 is currently in progress, we are in the middle of moving project and CI/CD to GitHub from private repository. Many TimeBase 6.0 features and many public materials are also work in progress

avavilau commented 4 years ago

Existing Materials:

avavilau commented 4 years ago

Development Team:

https://github.com/stuartfarr - Business https://github.com/avavilau - Delivery Manager https://github.com/alex-karpovich - Chief Solution Architect / Main Developer https://github.com/alexei-osipov - Solution Architect / Main Developer https://github.com/andymalakov - Solution Architect https://github.com/Romkisel - Backend Developer https://github.com/ypldan - Backend Developer https://github.com/Maxim-Gadalov - Backend Developer https://github.com/sizmestiev - C++ Developer https://github.com/MitoZ - Front-end Developer https://github.com/msamusenka - Solution Architect (Integrations) https://github.com/ekarankow - Backend Developer (Integrations) https://github.com/PolinaGurinovich - Front-end Developer (Integrations) https://github.com/astrakh - QA Automation https://github.com/sr-remsha - Technical Writer

avavilau commented 4 years ago

Contribution process

Contribution guidelines are work in progress and not yet defined.

ColinEberhardt commented 4 years ago

Thanks for raising this contribution proposal, TimeBase looks like an interesting and useful product for the FINOS members and the wider community.

Considering that this is already a commercial product, I would be grateful if you could elaborate on the differences between the open source community edition and the paid-for enterprise version? This type of arrangement can cause conflict, you will certainly wish to retain sales of your commercial edition, and as a result, I expect that you will 'hold back' certain product features. However, for the community edition to be a success (beyond being a teaser for the commercial edition), it needs to be a well-rounded and useful product in its own right.

Can you please share your thoughts on this?

stuartfarr commented 4 years ago

@ColinEberhardt, thank you for question; your comment is well made. You can see the differences here. But some additional comments and context: TimeBase's original mission was (and is) to provide high performance time-series data management for systematic/algorithmic trading firms as part of larger product suite we provide. Many such firms are small and want "out of the box" capability. As such, we bundled data (ingest) connectors for many trading venues and associated market data structures (streams). These are not part of TimeBase Community Edition because we want and expect a much wider set of use-cases which may or may not need these connectors. Another piece of relevant background is that TimeBase was built by a small software company called Deltix, acquired by EPAM in February 2020. We (old Deltix) had considered making an open source contribution years before but the support of EPAM has given us the confidence to finally do it! TimeBase Community is emphatically not a teaser for the commercial product which, outside of the differences noted, is the same core engine as the enterprise edition. This is an important topic, please let me know if this comment addresses it.

mcleo-d commented 4 years ago

Please find the differences between Community Edition and Enterprise Edition below ...

Screenshot 2020-10-26 at 18 00 51

avavilau commented 4 years ago

@mcleo-d repository: https://github.com/DeltixInc/TimeBase

avavilau commented 4 years ago
avavilau commented 4 years ago

TB Logo: https://timebase.info/assets/img/timebase-logo-new.svg

mcleo-d commented 3 years ago

Hey @stuartfarr - I'm currently updating the progression of the Timebase CE project contribution and was wondering if you have an update on inviting current Timebase customers to join the project contribution?

It would be great if you could supply an update in the issue comments below.

Kind regards,

James.

mcleo-d commented 3 years ago

Hi @stuartfarr, @avavilau and @alex-karpovich,

Can you ask the following Timebase team members to accept their invitation to the FINOS organisation on GitHub as part of the Timebase CE project contribution 👍

Many thanks,

James 🚀

stuartfarr commented 3 years ago

Hey @stuartfarr - I'm currently updating the progression of the Timebase CE project contribution and was wondering if you have an update on inviting current Timebase customers to join the project contribution?

It would be great if you could supply an update in the issue comments below.

Kind regards,

James.

There are a couple of new (TimeBase CE prospects) that I hoped we would be able to, but it looks like they are going to fall into 2021 now.

mindthegab commented 3 years ago

Thanks @stuartfarr and the team of contributors for putting this forward. I think we should proceed with the contribution validation and onboarding, and then use the socialization phase to garner more contributors.

@stuartfarr please let us know we can help once the project is contributed. Definitely existing prospects would be a great place to start to build a community.

Thanks!

mcleo-d commented 3 years ago

Thanks @mindthegab - I have setup a meeting with the TimeBase team for Monday 18th January at 2:30pm GMT to move forward with next steps 👍

James.

mcleo-d commented 3 years ago

The following update was received from @avavilau in the Timebase team ...

  • We’ve created all the repositories for all dependencies.
  • We’ve ported our CI scripts from gitlab pipelines to github workflows
  • We’ve started publishing artifacts to maven. (docker/artifacthub/nuget in progress)
  • We still need a bit more time to finish the process.

James.

stuartfarr commented 3 years ago

We are pleased to announce that our friends at crypto trading technology firm mainbloq will be the first consumer of TimeBase Community Edition.

mdvx commented 3 years ago

Absolutely thrilled to be working with TimeBase, we have big plans for this breakthrough technology. Can't wait to get started!

mcleo-d commented 3 years ago

Hi @mdvx - It's an absolute pleasure to meet you and look forward to working with you and the TimeBase team when the project hits the FINOS organisation 🚀

It would be great to catch up and talk about FINOS CCLAs so we can hit the ground running. Are you okay if @stuartfarr introduces us via email?

James.

cc @mindthegab @toshaellison @maoo @cm-howard

mdvx commented 3 years ago

Hi James

Nice to meet you, looking forward to working together. Let me also Introduce you to Clay Cowdery who will be taking an active role in this deployment.

All good with Stuart doing intros.

Marc

From: James McLeod @.> Sent: Thursday, April 1, 2021 6:51 AM To: finos/community @.> Cc: Marc Deveaux @.>; Mention @.> Subject: Re: [finos/community] TimeBase Community Edition (WIP) (#70)

Hi @mdvx https://github.com/mdvx - It's an absolute pleasure to meet you and look forward to working with you and the TimeBase team when the project hits the FINOS organisation 🚀

It would be great to catch up and talk about FINOS CCLAs so we can hit the ground running. Are you okay if @stuartfarr https://github.com/stuartfarr introduces us via email?

James.

cc @mindthegab https://github.com/mindthegab @toshaellison https://github.com/toshaellison @maoo https://github.com/maoo

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/finos/community/issues/70#issuecomment-811825315 , or unsubscribe https://github.com/notifications/unsubscribe-auth/AAUKUF3YVTTAMPHRBTB3GPTTGRFYVANCNFSM4R45EKGQ . https://github.com/notifications/beacon/AAUKUF2UWTODAQD2IG5AKJLTGRFYVA5CNFSM4R45EKG2YY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOGBRXRIY.gif

mdvx commented 3 years ago

Hi Guys

anybody got a URL for the docker image?

@stuartfarr @mcleo-d

mcleo-d commented 3 years ago

Hi @mdvx

Apologies for the time taken to respond. Let me email @avavilau and @stuartfarr to see how close the TimeBase team is to making the contribution and we'll get right back to you.

James.

mcleo-d commented 3 years ago

Hi @KimmoLinna

Thank you for meeting yesterday and for showing your initial interest in TimeBase CE as an open source alternative to KDB.

I'd like to introduce you to @stuartfarr and the TimeBase team who I'm sure will be in touch once the project is contributed to FINOS.

Have a great weekend and speak soon.

James.

stuartfarr commented 3 years ago

Hi @mdvx

Contribution to FINOS should be available in next 2 weeks: we will announce accordingly.

Thanks, Stuart

stuartfarr commented 3 years ago

Hi @kimmolinna

You may have seen that the TimeBase Community Edition (CE) contribution to FINOS is due early may.

In case you have not seen it, the site has information on TimeBase CE. But let me know if you have any questions.

Stuart

mcleo-d commented 3 years ago

Following a quick review of the TimeBase project prior to FINOS contribution, the issue below describes prerequisites needed within the open source project as outlined in the FINOS Contribution Compliance Requirements document.

stuartfarr commented 3 years ago

Hi @mcleo-d

Duly noted: TimeBase contribution will be ready 18th May but understood that subject to vulnerability scanning etc. for which we can provide @maoo the required access to the current EPAM/TimeBase repo.

mcleo-d commented 3 years ago

Hi @stuartfarr,

Thanks so much for your update. This is very exciting news! 🚀

Feel free to send me a link to the main Timebase repo when the project is ready for review.

James.

mcleo-d commented 3 years ago

@stuartfarr has confirmed TimeBase can move to the next stage of FINOS contribution alongside the completion of https://github.com/epam/TimeBase/issues/1 by the TimeBase team.

maoo commented 3 years ago

Hi @mcleo-d and @alex-karpovich ! I've raised https://github.com/finos/TimeBase-CE/pull/174 as I noticed that NOTICE file was still not correct.

Any update on the TimeBase logo, so that we can complete the project onboarding and complete the contribution process?

Thanks!

maoo commented 3 years ago

Quick status update - The TimeBase code is now hosted to https://github.com/finos/TimeBase-CE - congratulations to the EPAM team!

To complete the contribution process, we'd also like to:

  1. solve a logo visualization issue on https://landscape.finos.org/
  2. release on Maven Central and Docker (although not mandated by FINOS standard contribution process)
  3. LF Onboarding (@maoo)
    1. Enable LFX EasyCLA (EPAM already onboarded on EasyCLA)
    2. LFX Insights
    3. SFDC
maoo commented 3 years ago

Logo is fixed, see https://landscape.finos.org/

maoo commented 3 years ago

Project is registered in our internal systems, releases are done (soon README will be updated with badges).

Marking contribution as done, congrats to the EPAM team for this great contribution!