search

bisq-network / roles

@bisq-network contributor roles
https://bisq.wiki/Roles
15 stars 16 forks source link

Markets Website+API Operator/Maintainer #9

Open cbeams opened 7 years ago

cbeams commented 7 years ago

This role is responsible for operating the https://markets.bisq.network website and API as well as maintaining its source in the https://github.com/bisq-network/markets repository. Primary owner/maintainer: @wiz
Secondary owner/operator: @Emzy

cbeams commented 6 years ago

@dan-da, per bisq-network/dao#30, your dan-da/bitsquare_markets repo has been brought under the @bisq-network org at https://github.com/bisq-network/markets. Are you interested in continuing to fill the role of maintaining the codebase and operating the website? This would of course include getting compensated in $BSQ for your work.

If so, great. Please accept your invitation to join the @bisq-network organization. I'll get you assigned to this role, and we can get everything in order. If not, just let me know so we can put the word out about finding someone to take over this role. Thanks.

ManfredKarrer commented 6 years ago

2018.04 report

I am operating the market server, so as there is no one else responsible I will do the report: No known incidents occurred but I do not follow or monitor it closely.

ManfredKarrer commented 6 years ago

2018.05

Nothing to report

ManfredKarrer commented 6 years ago

2018.06 report

Update to 0.7.0.

ManfredKarrer commented 6 years ago

2018.07 report

Nothing to report.

/cc bisq-network/compensation#93

ManfredKarrer commented 6 years ago

2018.08 report

Nothing to report.

/cc bisq-network/compensation#112

ManfredKarrer commented 5 years ago

2018.09 report

Nothing to report.

/cc bisq-network/compensation#125

ManfredKarrer commented 5 years ago

2018.09 report

Nothing to report.

/cc bisq-network/compensation#125

ManfredKarrer commented 5 years ago

2018.10 report

Nothing to report.

/cc bisq-network/compensation#155

ManfredKarrer commented 5 years ago

2018.10 report

Nothing to report.

/cc bisq-network/compensation#155

ManfredKarrer commented 5 years ago

2018.11 report

Nothing to report.

/cc bisq-network/compensation#180

ManfredKarrer commented 5 years ago

2018.11 report

Updated to version 0.9.1 (required jdk update).

/cc bisq-network/compensation#189

ManfredKarrer commented 5 years ago

Issue with out of memory errors. Not clear why it happend as server reported sufficient free memory. I increased setting of maxMemory of app from 300 to 500 MB and updated code to 0.9.3. There was also a weird class not found error. With newly compiled 0.9.3 app was working again.

ManfredKarrer commented 5 years ago

2019.01 report

See comment above.

/cc bisq-network/compensation#205

Emzy commented 5 years ago

2019.02 report

I took over the operation of the markets website server.

Every second week there seems to be a server "crash" (out of memory). It seems to me the memory leak is in apache/php, more investigation needed. Restarting the Apache2 daemon fixes the problem every time.

I did a software update. But after the update still there was one out of memory problem.

/cc bisq-network/compensation#225

ManfredKarrer commented 5 years ago

2019.02 report

Nothing to report.

/cc bisq-network/compensation#227

Emzy commented 5 years ago

2019.03 report

Two times the server hat a out of memory problem. Restarting of apache and the bisq fixed it. I still have an eye on it.

/cc bisq-network/compensation#246

ManfredKarrer commented 5 years ago

2019.03 report

Nothing to report.

/cc bisq-network/compensation#252

Emzy commented 5 years ago

Cycle 1 report

The server hat no out of memory problem anymore.

/cc bisq-network/compensation#279

ManfredKarrer commented 5 years ago

@mrosseel Takes over primary role owner.

Emzy commented 5 years ago

Cycle2 report

The server had many out of memory problems again. Finally changed some parameters of the Apache2 service, seems to have worked so far.

/cc bisq-network/compensation#298

Emzy commented 5 years ago

Cycle3 report

The server had two out of memory problems again.

/cc bisq-network/compensation#310

Emzy commented 5 years ago

Cycle 4 report

The server more of memory problems again.

But not for 2 weeks. So I think I fixed it. Changed parameters of the Apache2 service again.

/cc bisq-network/compensation#324

Emzy commented 5 years ago

Cycle 5 report

The server is running fine after the last parameter fix. No restarts needed. Coordinating with the new maintainer.

/cc bisq-network/compensation#355

Emzy commented 4 years ago

Cycle 6 report

Everything running fine.

/cc bisq-network/compensation#373

wiz commented 4 years ago

Cycle 6 report

As of today I'm taking over the primary maintainer role from @mrosseel (secondary will remain with @Emzy), and I've migrated the markets.bisq.network server to its new location. Currently investigating how to scale the markets API to handle more requests, as the server gets heavily overloaded at times.

/cc https://github.com/bisq-network/compensation/issues/380

Emzy commented 4 years ago

Cycle 7 report

Secondary role. Got login for new server from @wiz

/cc bisq-network/compensation#397

wiz commented 4 years ago

Cycle 7 report

The Bisq Markets API server is under heavy load, but we're going to replace it soon with a full re-implementation using a smart-caching proxy and new API backend.

/cc bisq-network/compensation#398

wiz commented 4 years ago

Cycle 8 report

The Bisq Markets API has been migrated to a new implementation utilizing a smart-caching proxy on Google Cloud Appengine and new API backend powered by risq. The new service is much faster for serving chart data on bisq.network/markets and is serving millions of requests daily to external third parties who have integrated the API into their sites.

This month the expenses are a bit higher due to one-off Google Cloud costs, i.e. as part of the migration before we optimized the site to lower costs. Going forward the site should run less than $300 per month hosting cost, which is a good deal considering the huge amount of traffic it gets. Ideally we will move away from Google at some point in the future, but we would need a very beefy CDN setup of some kind. The good thing is that the service is at 100% uptime now and very fast performance!

/cc https://github.com/bisq-network/compensation/issues/448

wiz commented 4 years ago

Cycle 9 report

A few days ago we decided revert the Bisq Markets API to a fully self-hosted solution, and the temporary risq + google architecture has now been shut down. This decision was made since the risq project is no longer being maintained, and so I instead added some performance tweaks to the previous Bisq Statsnode + PHP service architecture.

I will continue to improve the Statsnode architecture, with the next goal being a one-command installer to make it easy to deploy multiple API service nodes and load-balance between them. This will allow us to maintain a self-hosted solution and scale it as Bisq grows.

/cc https://github.com/bisq-network/compensation/issues/470

wiz commented 4 years ago

Cycle 10 report

The one-command installer was added, and I used it to create a new ubuntu 18 php 7 based server for the Markets API service, running on an NVM backed datastore, which seems to have greatly improved performance with php 7's new opcache and diskcache extensions. If necessary I will add a few more of these instances and load balance between them later.

wiz commented 4 years ago

Cycle 11 report

Markets API is very smooth on the new setup, performance seems much faster and all issues resolved.

/cc bisq-network/compensation#515

cbeams commented 4 years ago

@Emzy is this still a role you're involved with?

Emzy commented 4 years ago

@Emzy is this still a role you're involved with?

Yes I'm backup Operator for @wiz.

wiz commented 4 years ago

Cycle 12 report

Markets API is stable on the setup, except for having to be restarted a few times due to the v1.3 upgrade related issues.

https://github.com/bisq-network/compensation/issues/547

wiz commented 4 years ago

Cycle 13 report

Markets API required a few restarts this cycle, sometimes getting stuck like Bisq occasionally does. Other than that, no issues.

https://github.com/bisq-network/compensation/issues/569

wiz commented 4 years ago

Cycle 14 report

Markets API is running stable on a new server

/cc https://github.com/bisq-network/compensation/issues/602

wiz commented 4 years ago

Cycle 15 report

The Markets API has been very congested recently due an increase in API requests. I heard from @m52go that the /markets page on the website is getting very popular for some reason. I plan to migrate the Markets API to a cluster of 2 new servers soon, which will add additional capacity and hopefully solve the performance issues. I also intend to serve it from the bisq.market domain name as per https://github.com/bisq-network/proposals/issues/229 once approved.

/cc https://github.com/bisq-network/compensation/issues/632

wiz commented 3 years ago

Cycle 16/17 report

It took a while, but we've finally migrated the Bisq Markets API away from the old LAMP architecture to its new optimized nginx+NodeJS architecture with a reverse proxy cache in front of 2 new servers running the completely new implementation by @softsimon as part of https://github.com/bisq-network/projects/issues/41 - performance on the https://bisq.network/markets website is now fast again, and the ops team budget has been updated for the dual node setup.

The Markets API endpoint hostname has also been migrated from markets.bisq.network/api to bisq.markets/api so this change should be communicated to users of the API, possibly over Twitter and direct communication to them if we have it. The new bisq.markets website will have improved API documentation, but for now it just hosts the previous static HTML file as a placeholder until that project is ready to launch.

We're now beginning the new Angular based webapp frontend development phase of https://github.com/bisq-network/projects/issues/41 for the new bisq.markets website project - if anyone has any features they want implemented feel free to comment on the project issue.

wiz commented 3 years ago

Cycle 25 report

wiz commented 6 months ago

I'm resigning as Bisq Markets operator and will transfer the bisq.markets domain name to @Emzy so he can takeover

wiz commented 5 months ago

I've ceased operating bisq.markets and will 301 redirect it to bisq.mempool.emzy.de

cbeams commented 5 months ago

Could you do a 302 instead, please?On Apr 2, 2024, at 8:51 AM, wiz @.***> wrote: I've ceased operating bisq.markets and will 301 redirect it to bisq.mempool.emzy.de

—Reply to this email directly, view it on GitHub, or unsubscribe.You are receiving this because you authored the thread.Message ID: @.***>