Flux, Your Gateway to a Decentralized World. https://home.runonflux.io https://api.runonflux.io https://docs.runonflux.io https://source.runonflux.io https://wiki.runonflux.io
Stop broadcasting over the network every 1h apps running on each node;
Reduce almost all the bandwidth used by fluxos as currently they require the appsrunning messages to be broadcasted continuasly to know where are the apps running on the network;
Reduce the time to start trying to install apps;
I will put much more information on the PR once it's finished.
List of changes:
Config/default.js ->
apprunningRefactorActivation - block when the changes that require a fork will activate;
appsRunningTimestampRestore - where we are going to store on db the information if a specific timestamp should be used to sync appsLocations.
src/routes.js ->
new api method '/apps/broadcastappsinstalled' - the node that receive it will broadcast to the network the list of apps instaleld;
new api method '/apps/locationscompressed' that returns the information from apps location db compressed;
new api method '/apps/installedappsnames/:url?' that returns the list of app names installed from the node itself or from the node in the url;
src/services/appsService.js ->
storeAppRunningMessage method - apprunning message v2 will now start to be used to send information of all apps running where v1 is sent when one app is installed, so, when we receive v2 message, we check on db if there was app name on that ip that was not broadcasted with the message and we set as removed if that is the case.
Another change on this method is the use of FluxService.canProcessAppsRunningMessages() to know if we can store the message on db, it will be explained further;
storeAppRemovedMessage method - use of FluxService.canProcessAppsRunningMessages() to know if we can store the message on db, it will be explained further.
Do not delete information from db, instead, update row adding removedBroadcastedAt when message was received and expiration.
reindexGlobalAppsLocation method - new index creation adding removedBroadcastAt;
appLocation method - add db filter where removedBroadcastAt is null;
new method getAppsLocationsDB - this method will be used by other nodes when they are starting the fluxOs to sync the appsLocations DB;
getRunningAppIpList method - add db filter where removedBroadcastAt is null;
trySpawningGlobalApplication method - use of FluxService.canProcessAppsRunningMessages() to know if we can try to spawn a app on the node, it will be explained further.
checkAndNotifyPeersOfRunningApps method - after apprunningRefactorActivation only allow peers notifications once every 48h.
Start sending on this method always the v2 appsrunning message;
new method broadcastAppsInstalled - method called by other nodes when they find difference on what this node have installed vs what was on their database, the node will broadcaste a v2 apprunning message to the network saying what is installed/running;
new method removeAppsRunningOnNodeIP - method that sets removedBroadcasted twhen a new message nodedown is received;
src/services/dbHelper.js ->
new method limitFromCollection -> Returns the Max or the Minimum value of the sort query:
src/services/explorerService.js ->
method processBlock - on apprunningRefactorActivation block activation we drop index broadcastedAt on appslocations because it had 'expireAfterSeconds' and recreate it without that expire propery;
initiateBlockProcessor method - we add a new index with removeBroadcastedAt on appslocation;
src/services/fluxCommunication.js ->
new method handleNodeDownMessage - method responsable to handle the new message nodedown on the network;
methods handleIncomingConnection and initiateAndHandleConnection - handle the new message nodedown;
src/services/fluxServices.js ->
new method monitorAppsRunningOnNodesDoubleCheck - this method will double check after few minutes if node is down/not reachable and if that is the case will broadcast nodedown message to the network. If node is not down it will check if apps installed on the node match the local db, if not it calls node api to broadcast the apps installed;
new method monitorAppsRunningOnNodes - it will select a random node on the network and check if the node apps installed answer match the local db information, if not asks the node to broadcast the apps installed over the network;
new method prepareAppsLocationsDB - This method is the responsable to sync the appslocations DB on fluxOs startup;
We check on db if the method was started before without finishing properly, if there is on db a time that should be used for the sync that is what will be used.
If the local information don't have any update in the last 24h, we delete the entire db and ask another random node on the network with some uptime for it, if the local db have updates with less than 24h, we grab the latest update timestamp, and reduce it one hour and use it to ask another node just the updates since that timestamp.
Because there is some cache information on apis, once the update is done, we await some seconds and we ask again other nodes the latest updates since the prepareAppsLocationsDB has started.
In this method we set enableProcessingAppsRunningMessages property that is returned on FluxService.canProcessAppsRunningMessages(). We will not store appslocations information when the DB is in the initial sync process because it could lead to just grabbing latest seconds apps locations if a message was received and the db was fresh/out of sync.
src/services/serviceManager.js ->
method startFluxFunctions - change index creations/drops regarding appslocations because of the expireAfterSeconds.
Start fluxService.prepareAppsLocationsDB().
Change checkAndNotifyPeersOfRunningApps call to be every 48h after apprunningRefactorActivation.
Start appsService.trySpawningGlobalApplication(); after 20m of FluxOs start instead of 125m.
Call fluxService.monitorAppsRunningOnNodes() every 5 minutes.
This PR is not finished - WIP - Missing tests
I will put much more information on the PR once it's finished. List of changes: Config/default.js ->
src/routes.js ->
src/services/appsService.js ->
src/services/dbHelper.js ->
src/services/explorerService.js ->
src/services/fluxCommunication.js ->
src/services/fluxServices.js ->
src/services/serviceManager.js ->