search

ortexx / museria-global

Global decentralized music storage
MIT License
11 stars 4 forks source link

Not found a suitable server to store the song #4

Closed Vort closed 4 years ago

Vort commented 4 years ago

When I try to upload song via my own node, I am getting "Not found a suitable server to store the song" error. Also search is not working - songs, which can be found via storage.museria.com, are not found via my node.

It is possible that node is wrongly configured or built. But it is impossible to tell because documentation and software logs do not contains needed details.

museria_server

ortexx commented 4 years ago

You can find information in the docs that your node must be open for external requests. So, if you run it in localhost you can't connect to the global network. But if you want just see how it works on the localhost and create your test network then pass node arguments --hostname=localhost --initialNetworkAddress=localhost:2079 manually.

npm start -- --hostname=localhost --initialNetworkAddress=localhost:2079

All these options are described in the spreadable readme.

Vort commented 4 years ago

I was not able start it without port forwarding. So it is definitely open for external requests.

Vort commented 4 years ago

Maybe it is a problem? I see such error spam in console:

Error: Ip "45.138.158.37" is in the banlist
    at NodeMuseriaGlobal.addressFilter (c:\museria\node_modules\spreadable\src\n
ode.js:1253:15)
    at async NodeMuseriaGlobal.request (c:\museria\node_modules\spreadable\src\n
ode.js:1386:7)
    at async NodeMuseriaGlobal.requestServer (c:\museria\node_modules\spreadable
\src\node.js:1559:22)
    at async NodeMuseriaGlobal.normalizeRoot (c:\museria\node_modules\spreadable
\src\node.js:994:24)
    at async NodeMuseriaGlobal.sync (c:\museria\node_modules\spreadable\src\node
.js:458:7)
    at async NodeMuseriaGlobal.sync (c:\museria\node_modules\metastocle\src\node
.js:84:7)
    at async NodeMuseriaGlobal.sync (c:\museria\node_modules\storacle\src\node.j
s:151:7)
    at async Timeout.fn [as _onTimeout] (c:\museria\node_modules\spreadable\src\
node.js:137:11)
ortexx commented 4 years ago

Yea, it is a problem. Remove ./museria/loki.db file and restart your node. But be sure that everything with the external requests is good now .

Vort commented 4 years ago

No "ban" messages after loki.db removal. But still no upload possibility. Right now I forward 2079 port. Is anything else needed?

ortexx commented 4 years ago

Show me your server ip address

Vort commented 4 years ago

But I will remove it soon. Do not want search engines to index it. http://hide/app

ortexx commented 4 years ago

I edited. Your server is not joined to the network. Can you please:

Vort commented 4 years ago 
c:\museria>node index.js --storageDataSize=1gb --storageTempSize=1gb
Museria global node has been launched
MUSERIA_HOSTNAME=
MUSERIA_PORT=2079
MUSERIA_PUBLIC_PORT=2079
MUSERIA_INITIAL_NETWORK_ADDRESS=storage.museria.com:80
MUSERIA_LOGGER_LEVEL=error
MUSERIA_COLLECTION_MUSIC_LIMIT=auto
MUSERIA_STORAGE_PATH=./museria
MUSERIA_STORAGE_DATA_SIZE=95% - 2gb
MUSERIA_STORAGE_TEMP_SIZE=2gb
{"filename":"museria\\loki.db","collections":[{"name":"cache","data":[],"idIndex":[],"binaryIndices":{"type":{"name":"type","dirty":false,"values":[]}},"constraints":null,"uniqueNames":[],"transforms":{},"objType":"cache","dirty":false,"cachedIndex":null,"cachedBinaryIndex":null,"cachedData":null,"adaptiveBinaryIndices":true,"transactional":false,"cloneObjects":false,"cloneMethod":"parse-stringify","asyncListeners":false,"disableMeta":true,"disableChangesApi":true,"disableDeltaChangesApi":true,"autoupdate":false,"serializableIndices":true,"ttl":null,"maxId":0,"DynamicViews":[],"events":{"insert":[],"update":[],"pre-insert":[],"pre-update":[],"close":[],"flushbuffer":[],"error":[],"delete":[null],"warning":[null]},"changes":[],"dirtyIds":[]},{"name":"data","data":[{"name":"rootNetworkAddress","value":"storage.museria.com:80","$loki":1},{"name":"registrationTime","value":null,"$loki":2},{"name":"checkedMasterStructures","value":[],"$loki":3},{"name":"filesTotalSize","value":0,"$loki":4},{"name":"filesCount","value":0,"$loki":5}],"idIndex":[1,2,3,4,5],"binaryIndices":{},"constraints":null,"uniqueNames":["name"],"transforms":{},"objType":"data","dirty":false,"cachedIndex":null,"cachedBinaryIndex":null,"cachedData":null,"adaptiveBinaryIndices":true,"transactional":false,"cloneObjects":false,"cloneMethod":"parse-stringify","asyncListeners":false,"disableMeta":true,"disableChangesApi":true,"disableDeltaChangesApi":true,"autoupdate":false,"serializableIndices":true,"ttl":null,"maxId":5,"DynamicViews":[],"events":{"insert":[],"update":[],"pre-insert":[],"pre-update":[],"close":[],"flushbuffer":[],"error":[],"delete":[null],"warning":[null]},"changes":[],"dirtyIds":[]},{"name":"servers","data":[],"idIndex":[],"binaryIndices":{},"constraints":null,"uniqueNames":["address"],"transforms":{},"objType":"servers","dirty":false,"cachedIndex":null,"cachedBinaryIndex":null,"cachedData":null,"adaptiveBinaryIndices":true,"transactional":false,"cloneObjects":false,"cloneMethod":"parse-stringify","asyncListeners":false,"disableMeta":true,"disableChangesApi":true,"disableDeltaChangesApi":true,"autoupdate":false,"serializableIndices":true,"ttl":null,"maxId":0,"DynamicViews":[],"events":{"insert":[],"update":[],"pre-insert":[],"pre-update":[],"close":[],"flushbuffer":[],"error":[],"delete":[null],"warning":[null]},"changes":[],"dirtyIds":[]},{"name":"banlist","data":[{"createdAt":1585754604769,"updatedAt":1585754604769,"reason":"registration","address":"144.91.66.249:2079","ip":"0000:0000:0000:0000:0000:ffff:905b:42f9","resolvedAt":1588087404769,"$loki":1},{"createdAt":1585754639422,"updatedAt":1585754639422,"reason":"registration","address":"storage.museria.com:80","ip":"0000:0000:0000:0000:0000:ffff:2d8a:9e25","resolvedAt":1588087439422,"$loki":2}],"idIndex":[1,2],"binaryIndices":{},"constraints":null,"uniqueNames":["address"],"transforms":{},"objType":"banlist","dirty":false,"cachedIndex":null,"cachedBinaryIndex":null,"cachedData":null,"adaptiveBinaryIndices":true,"transactional":false,"cloneObjects":false,"cloneMethod":"parse-stringify","asyncListeners":false,"disableMeta":true,"disableChangesApi":true,"disableDeltaChangesApi":true,"autoupdate":false,"serializableIndices":true,"ttl":null,"maxId":2,"DynamicViews":[],"events":{"insert":[],"update":[],"pre-insert":[],"pre-update":[],"close":[],"flushbuffer":[],"error":[],"delete":[null],"warning":[null]},"changes":[],"dirtyIds":[]},{"name":"approval","data":[],"idIndex":[],"binaryIndices":{"type":{"name":"type","dirty":false,"values":[]},"clientIp":{"name":"clientIp","dirty":false,"values":[]},"action":{"name":"action","dirty":false,"values":[]}},"constraints":null,"uniqueNames":[],"transforms":{},"objType":"approval","dirty":false,"cachedIndex":null,"cachedBinaryIndex":null,"cachedData":null,"adaptiveBinaryIndices":true,"transactional":false,"cloneObjects":false,"cloneMethod":"parse-stringify","asyncListeners":false,"disableMeta":true,"disableChangesApi":true,"disableDeltaChangesApi":true,"autoupdate":false,"serializableIndices":true,"ttl":null,"maxId":0,"DynamicViews":[],"events":{"insert":[],"update":[],"pre-insert":[],"pre-update":[],"close":[],"flushbuffer":[],"error":[],"delete":[null],"warning":[null]},"changes":[],"dirtyIds":[]},{"name":"behaviorCandidates","data":[],"idIndex":[],"binaryIndices":{"address":{"name":"address","dirty":false,"values":[]},"action":{"name":"action","dirty":false,"values":[]}},"constraints":null,"uniqueNames":[],"transforms":{},"objType":"behaviorCandidates","dirty":false,"cachedIndex":null,"cachedBinaryIndex":null,"cachedData":null,"adaptiveBinaryIndices":true,"transactional":false,"cloneObjects":false,"cloneMethod":"parse-stringify","asyncListeners":false,"disableMeta":true,"disableChangesApi":true,"disableDeltaChangesApi":true,"autoupdate":false,"serializableIndices":true,"ttl":null,"maxId":0,"DynamicViews":[],"events":{"insert":[],"update":[],"pre-insert":[],"pre-update":[],"close":[],"flushbuffer":[],"error":[],"delete":[null],"warning":[null]},"changes":[],"dirtyIds":[]},{"name":"behaviorDelays","data":[],"idIndex":[],"binaryIndices":{"address":{"name":"address","dirty":false,"values":[]},"action":{"name":"action","dirty":false,"values":[]}},"constraints":null,"uniqueNames":[],"transforms":{},"objType":"behaviorDelays","dirty":false,"cachedIndex":null,"cachedBinaryIndex":null,"cachedData":null,"adaptiveBinaryIndices":true,"transactional":false,"cloneObjects":false,"cloneMethod":"parse-stringify","asyncListeners":false,"disableMeta":true,"disableChangesApi":true,"disableDeltaChangesApi":true,"autoupdate":false,"serializableIndices":true,"ttl":null,"maxId":0,"DynamicViews":[],"events":{"insert":[],"update":[],"pre-insert":[],"pre-update":[],"close":[],"flushbuffer":[],"error":[],"delete":[null],"warning":[null]},"changes":[],"dirtyIds":[]},{"name":"behaviorFails","data":[],"idIndex":[],"binaryIndices":{"address":{"name":"address","dirty":false,"values":[]},"action":{"name":"action","dirty":false,"values":[]}},"constraints":null,"uniqueNames":[],"transforms":{},"objType":"behaviorFails","dirty":false,"cachedIndex":null,"cachedBinaryIndex":null,"cachedData":null,"adaptiveBinaryIndices":true,"transactional":false,"cloneObjects":false,"cloneMethod":"parse-stringify","asyncListeners":false,"disableMeta":true,"disableChangesApi":true,"disableDeltaChangesApi":true,"autoupdate":false,"serializableIndices":true,"ttl":null,"maxId":3,"DynamicViews":[],"events":{"insert":[],"update":[],"pre-insert":[],"pre-update":[],"close":[],"flushbuffer":[],"error":[],"delete":[null],"warning":[null]},"changes":[],"dirtyIds":[]},{"name":"metaMusic","data":[],"idIndex":[],"binaryIndices":{},"constraints":null,"uniqueNames":["title"],"transforms":{},"objType":"metaMusic","dirty":false,"cachedIndex":null,"cachedBinaryIndex":null,"cachedData":null,"adaptiveBinaryIndices":true,"transactional":false,"cloneObjects":false,"cloneMethod":"parse-stringify","asyncListeners":false,"disableMeta":true,"disableChangesApi":true,"disableDeltaChangesApi":true,"autoupdate":false,"serializableIndices":true,"ttl":null,"maxId":0,"DynamicViews":[],"events":{"insert":[],"update":[],"pre-insert":[],"pre-update":[],"close":[],"flushbuffer":[],"error":[],"delete":[null],"warning":[null]},"changes":[],"dirtyIds":[]}],"databaseVersion":1.5,"engineVersion":1.5,"autosave":true,"autosaveInterval":3000,"autosaveHandle":null,"throttledSaves":true,"options":{"filename":"museria\\loki.db","autosaveInterval":3000,"metaPrefix":"meta","autoload":true,"autosave":true,"serializationMethod":"normal","destructureDelimiter":"$<\n","recursiveWait":true,"recursiveWaitLimit":false,"recursiveWaitLimitDuration":2000,"started":1585754150346},"persistenceMethod":"fs","persistenceAdapter":null,"verbose":false,"events":{"init":[null],"loaded":[],"flushChanges":[],"close":[],"changes":[],"warning":[]},"ENV":"NODEJS"}
ortexx commented 4 years ago

The network nodes can't register your node. I haven't faced this behavior.

Could you:

Vort commented 4 years ago

I see

Error: Interviewee unavailable
    at NodeMuseriaGlobal.request (c:\museria\node_modules\spreadable\src\node.js
:1442:15)
    at processTicksAndRejections (internal/process/task_queues.js:97:5)
    at async NodeMuseriaGlobal.requestServer (c:\museria\node_modules\spreadable
\src\node.js:1559:22)
    at async NodeMuseriaGlobal.requestNode (c:\museria\node_modules\spreadable\s
rc\node.js:1510:14)
    at async NodeMuseriaGlobal.register (c:\museria\node_modules\spreadable\src\
node.js:745:18)
    at async NodeMuseriaGlobal.sync (c:\museria\node_modules\spreadable\src\node
.js:491:9)
    at async NodeMuseriaGlobal.sync (c:\museria\node_modules\metastocle\src\node
.js:84:7)
    at async NodeMuseriaGlobal.sync (c:\museria\node_modules\storacle\src\node.j
s:151:7)
    at async fn (c:\museria\node_modules\spreadable\src\node.js:137:11)
    at async NodeMuseriaGlobal.init (c:\museria\node_modules\spreadable\src\node
.js:145:7)

Node.JS problem again?

Vort commented 4 years ago

One more:

Error: Network hasn't been normalized yet, try later
    at NodeMuseriaGlobal.register (c:\museria\node_modules\spreadable\src\node.j
s:721:15)
    at processTicksAndRejections (internal/process/task_queues.js:97:5)
    at async NodeMuseriaGlobal.sync (c:\museria\node_modules\spreadable\src\node
.js:491:9)
    at async NodeMuseriaGlobal.sync (c:\museria\node_modules\metastocle\src\node
.js:84:7)
    at async NodeMuseriaGlobal.sync (c:\museria\node_modules\storacle\src\node.j
s:151:7)
    at async Timeout.fn [as _onTimeout] (c:\museria\node_modules\spreadable\src\
node.js:137:11)
ortexx commented 4 years ago

Error: Interviewee unavailable - is the reason, but it is really strange Network hasn't been normalized yet, try later - it is ok, just normal warning

Vort commented 4 years ago

You can look at that machine via TeamViewer 14: ID: [redacted] Pass: 4597 But try not to destroy it ) And not wonder if you find something strange there )

ortexx commented 4 years ago

i found out the reason. Server in your machine recognizes client ip address as 10.0.2.2. There are three possible options:

So, i need to figure it out.

Vort commented 4 years ago

As you may noticed, this box is virtual machine. And I don't have public IP for it. Instead I use NAT to forward port access from my public IP to VM internal IP.

Same thing usually happens when users have Internet connected via router, so such configuration is not rare.

ortexx commented 4 years ago

Yes, but we need to get the real client ip address some way, if it is possible. I will try to find more information about it.

In typical case there wouldn't be such issue.

Vort commented 4 years ago

Yes, but we need to get the real client ip address some way

  1. It can be configured.
  2. Another node can tell it.

And I saw in some messages that client knows my real IP.

Vort commented 4 years ago

You can replicate such configuration on your PC with VirtualBox and even Linux as guest. Just add forwarding to VM settings: https://i.stack.imgur.com/mfCcF.jpg

ortexx commented 4 years ago

is there any way to fill x-forwarded-for header with the client ip address as well? I am not sure that it is possible to fetch it otherwise, using nodejs tools.

Vort commented 4 years ago

Looks like X-Forwarded-For is for reverse situation, when internal IP is hidden, but server needs to know it. I know nothing how client is made, but it should be possible to transmit any data between nodes. With GET, POST, Headers, body etc.

If you asking how to get client IP at server side, it should be not too hard: https://stackoverflow.com/questions/8107856/how-to-determine-a-users-ip-address-in-node

ortexx commented 4 years ago

Examples by your link won't work. It is the problem. I do the same in the code, but we can't get the real client(user) ip inside the container without filling x-forwarded-for header before. With nginx, for example.

Tomorrow i will go deep into it)

Vort commented 4 years ago

At TCP level server knows real (public) IP of remote user. As I understand your message, it is a problem to propagate this information to the needed place.

It is time for me to go to sleep too :)

ortexx commented 4 years ago

I updated the readme https://github.com/ortexx/museria-global#what-are-the-requirements

Vort commented 4 years ago

VBoxManage modifyvm "Windows 7" --nataliasmode1 proxyonly solved the problem. Thanks.

ortexx commented 4 years ago

Yea, i can see your node now. Thank you for the feedback!