Closed AmineDiro closed 3 years ago
Hi @AmineDiro. I am unsure if we have had any users successfully set up fiftyone
on RHEL. But it is something we can try to work out.
The first thing I would recommend is to clear your mongo data folder, and then try again with your system install of mongo.
rm -r /users/l0494241/.fiftyone/var/lib/mongo
Thanks a lot for your help, i rm - rf I am still having the same issue.
From my understanding the init calls __public__
which initalizes the DatabaseService
class.
In the class we call `DatabaseService.find_mongod()``which search for available mongod_path .
For me i debugged by printing the mongod_path
mogod_path /data_local/workspace/conda_amine/lib/python3.7/site-packages/fiftyone/db/bin/mongod
mogod_path /usr/bin/mongod
Now after rerunning i have this error :
{"t":{"$date":"2021-07-02T16:17:00.976Z"},"s":"I", "c":"CONTROL", "id":20697, "ctx":"main","msg":"Renamed existing log file","attr":{"oldLogPath":"/users/l0494241/.fiftyone/var/lib/mongo/log/mongo.log","newLogPath":"/users/l0494241/.fiftyone/var/lib/mongo/log/mongo.log.2021-07-02T16-17-00"}}
Subprocess ['/usr/bin/mongod', '--dbpath', '/users/l0494241/.fiftyone/var/lib/mongo', '--logpath', '/users/l0494241/.fiftyone/var/lib/mongo/log/mongo.log'] exited with error 1:
}
Frame: {"a":"55C41D55F94E","b":"55C41C598000","o":"FC794E","s":"_start","s+":"29"},"s":"_ZN5mongo8executor18NetworkInterfaceTLC1ENSt7__cxx1112basic_stringIcSt11char_traitsIcESaIcEEENS0_14ConnectionPool7OptionsEPNS_14ServiceContextESt10unique_ptrINS0_21NetworkConnectionHookESt14default_deleteISD_EESC_INS_3rpc18EgressMetadataHookESE_ISI_EE","s+":"E88"}
Frame: {"a":"55C41ECB47CD","b":"55C41C598000","o":"271C7CD","s":"_ZN5mongo8executor20makeNetworkInterfaceENSt7__cxx1112basic_stringIcSt11char_traitsIcESaIcEEESt10unique_ptrINS0_21NetworkConnectionHookESt14default_deleteIS8_EES7_INS_3rpc18EgressMetadataHookES9_ISD_EENS0_14ConnectionPool7OptionsE","s+":"18D"}
Frame: {"a":"55C41D563565","b":"55C41C598000","o":"FCB565","s":"_ZN5mongo12_GLOBAL__N_116setUpReplicationEPNS_14ServiceContextE","s+":"405"}
Frame: {"a":"55C41D563FA2","b":"55C41C598000","o":"FCBFA2","s":"_ZN5mongo12_GLOBAL__N_111mongoDbMainEiPPcS2_","s+":"2C2"}
Frame: {"a":"55C41D4D7029","b":"55C41C598000","o":"F3F029","s":"main","s+":"9"}
Frame: {"a":"7F8F7D4463D5","b":"7F8F7D424000","o":"223D5","s":"__libc_start_main","s+":"F5"E.constprop.606","s+":"1EA"}
Frame: {"a":"55C41F32AD4A","b":"55C41C598000","o":"2D92D4A","s":"_ZN5mongo15printStackTraceERSo","s+":"3A"}
Frame: {"a":"55C41F2F3D4D","b":"55C41C598000","o":"2D5BD4D","s":"_ZN5mongo5logv214FileRotateSink7consumeERKN5boost3log12v2s_mt_posix11record_viewERKNSt7__cxx1112basic_stringIcSt11char_traitsIcESaIcEEE","s+":"50D"}
Frame: {"a":"55C41F30B4E3","b":"55C41C598000","o":"2D734E3","s":"_ZN5boost3log12v2s_mt_posix5sinks13unlocked_sinkIN5mongo5logv216CompositeBackendIJNS5_14FileRotateSinkENS5_10RamLogSinkES8_NS5_14UserAssertSinkEEEEE7consumeERKNS1_11record_viewE","s+":"213"}
Frame: {"a":"55C41F440223","b":"55C41C598000","o":"2EA8223","s":"_ZN5boost3log12v2s_mt_posix4core16push_record_moveERNS1_6recordE","s+":"1D3"}
Frame: {"a":"55C41F2FF3E2","b":"55C41C598000","o":"2D673E2","s":"_ZN5mongo5logv26detail9doLogImplEiRKNS0_11LogSeverityERKNS0_10LogOptionsENS_10StringDataERKNS0_26TypeErasedAttributeStorageE","s+":"682"}
Frame: {"a":"55C41ECCE138","b":"55C41C598000","o":"2736138"plicationEPNS_14ServiceContextE","s+":"405"},{"a":"55C41D563FA2","b":"55C41C598000","o":"FCBFA2","s":"_ZN5mongo12_GLOBAL__N_111mongoDbMainEiPPcS2_","s+":"2C2"},{"a":"55C41D4D7029","b":"55C41C598000","o":"F3F029","s":"main","s+":"9"},{"a":"7F8F7D4463D5","b":"7F8F7D424000","o":"223D5","s":"__libc_start_main","s+":"F5"},{"a":"55C41D55F94E","b":"55C41C598000","o":"FC794E","s":"_start","s+":"29"}],"processInfo":{"mongodbVersion":"4.4.6","gitVersion":"72e66213c2c3eab37d9358d5e78ad7f5c1d0d0d7","compiledModules":[],"uname":{"sysname":"Linux","release":"3.10.0-862.14.4.el7.x86_64","version":"#1 SMP Fri Sep 21 09:07:21 UTC 2018","machine":"x86_64"},"somap":[{"b":"55C41C598000","elfType":3,"buildId":"312311B4128B98F2F0502772FB85CC572CDDD0A1"},{"b":"7F8F7D424000","path":"/lib64/libc.so.6","elfType":3,"buildId":"85EA0AE559B53AB60D8548242CEDD0E83F4816DA"}]}}
Also after reinstalling mongod-server i could not start the service :
● mongod.service - MongoDB Database Server
Loaded: loaded (/usr/lib/systemd/system/mongod.service; enabled; vendor preset: disabled)
Active: failed (Result: exit-code) since Fri 2021-07-02 18:19:53 CEST; 4s ago
Docs: https://docs.mongodb.org/manual
Process: 26387 ExecStart=/usr/bin/mongod $OPTIONS (code=exited, status=14)
Process: 26384 ExecStartPre=/usr/bin/chmod 0755 /var/run/mongodb (code=exited, status=0/SUCCESS)
Process: 26380 ExecStartPre=/usr/bin/chown mongod:mongod /var/run/mongodb (code=exited, status=0/SUCCESS)
Process: 26376 ExecStartPre=/usr/bin/mkdir -p /var/run/mongodb (code=exited, status=0/SUCCESS)
Main PID: 15578 (code=exited, status=0/SUCCESS)
Jul 02 18:19:53 xrai-frpau-132 systemd[1]: Starting MongoDB Database Server...
Jul 02 18:19:53 xrai-frpau-132 mongod[26387]: about to fork child process, waiting until server is ready for connections.
Jul 02 18:19:53 xrai-frpau-132 mongod[26387]: forked process: 26390
Jul 02 18:19:53 xrai-frpau-132 mongod[26387]: ERROR: child process failed, exited with 14
Jul 02 18:19:53 xrai-frpau-132 mongod[26387]: To see additional information in this output, start without the "--fork" option.
Jul 02 18:19:53 xrai-frpau-132 systemd[1]: mongod.service: control process exited, code=exited status=14
Jul 02 18:19:53 xrai-frpau-132 systemd[1]: Failed to start MongoDB Database Server.
Jul 02 18:19:53 xrai-frpau-132 systemd[1]: Unit mongod.service entered failed state.
Jul 02 18:19:53 xrai-frpau-132 systemd[1]: mongod.service failed.
I am pretty stuck at this level , running mongod with --dbpath does not work either :/ .
Well , I found the issue with the mongod server not running it was a port issue, now i changed it to runn on 8889 , i also changed the value in command property of DatabaseService
:
262 @property
263 def command(self):
264 args = [
265 DatabaseService.find_mongod(),
266 "--dbpath",
267 self.database_dir,
268 "--logpath",
269 os.path.join(self.database_dir, "log/mongo.log"),
270 "--port",
271 "8889",
272 ]
Now the mongod server is running 👍
-bash-4.2$ dzdo systemctl status mongod
● mongod.service - MongoDB Database Server
Loaded: loaded (/usr/lib/systemd/system/mongod.service; enabled; vendor preset: disabled)
Active: active (running) since Fri 2021-07-02 19:33:14 CEST; 9min ago
Docs: https://docs.mongodb.org/manual
Main PID: 1797 (mongod)
CGroup: /system.slice/mongod.service
└─1797 /usr/bin/mongod -f /etc/mongod.conf
Jul 02 19:33:12 xrai-frpau-132 systemd[1]: Starting MongoDB Database Server...
Jul 02 19:33:13 xrai-frpau-132 mongod[1794]: about to fork child process, waiting until server is ready for connections.
Jul 02 19:33:13 xrai-frpau-132 mongod[1794]: forked process: 1797
Jul 02 19:33:14 xrai-frpau-132 mongod[1794]: child process started successfully, parent exiting
Jul 02 19:33:14 xrai-frpau-132 systemd[1]: Started MongoDB Database Server.
But there is still the same issue when importing the the DatabaseService 😞 :
Python 3.7.10 (default, Jun 4 2021, 14:48:32)
[GCC 7.5.0] :: Anaconda, Inc. on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import fiftyone as fo
mogod_path /data_local/workspace/conda_amine/lib/python3.7/site-packages/fiftyone/db/bin/mongod
mogod_path /usr/bin/mongod
{"t":{"$date":"2021-07-02T17:39:48.743Z"},"s":"I", "c":"CONTROL", "id":20697, "ctx":"main","msg":"Renamed existing log file","attr":{"oldLogPath":"/users/l0494241/.fiftyone/var/lib/mongo/log/mongo.log","newLogPath":"/users/l0494241/.fiftyone/var/lib/mongo/log/mongo.log.2021-07-02T17-39-48"}}
Subprocess ['/usr/bin/mongod', '--dbpath', '/users/l0494241/.fiftyone/var/lib/mongo', '--logpath', '/users/l0494241/.fiftyone/var/lib/mongo/log/mongo.log', '--port', '8889', '--nounixsocket'] exited with error 1:
}
Frame: {"a":"55737546E94E","b":"5573744A7000","o":"FC794E","s":"_start","s+":"29"},"s":"_ZN5mongo8executor18NetworkInterfaceTLC1ENSt7__cxx1112basic_stringIcSt11char_traitsIcESaIcEEENS0_14ConnectionPool7OptionsEPNS_14ServiceContextESt10unique_ptrINS0_21NetworkConnectionHookESt14default_deleteISD_EESC_INS_3rpc18EgressMetadataHookESE_ISI_EE","s+":"E88"}
Frame: {"a":"557376BC37CD","b":"5573744A7000","o":"271C7CD","s":"_ZN5mongo8executor20makeNetworkInterfaceENSt7__cxx1112basic_stringIcSt11char_traitsIcESaIcEEESt10unique_ptrINS0_21NetworkConnectionHookESt14default_deleteIS8_EES7_INS_3rpc18EgressMetadataHookES9_ISD_EENS0_14ConnectionPool7OptionsE","s+":"18D"} Frame: {"a":"557375472565","b":"5573744A7000","o":"FCB565","s":"_ZN5mongo12_GLOBAL__N_116setUpReplicationEPNS_14ServiceContextE","s+":"405"}
Frame: {"a":"557375472FA2","b":"5573744A7000","o":"FCBFA2","s":"_ZN5mongo12_GLOBAL__N_111mongoDbMainEiPPcS2_","s+":"2C2"}
Frame: {"a":"5573753E6029","b":"5573744A7000","o":"F3F029","s":"main","s+":"9"}
Frame: {"a":"7EFCED2E33D5","b":"7EFCED2C1000","o":"223D5","s":"__libc_start_main","s+":"F5"E.constprop.606","s+":"1EA"}
Frame: {"a":"557377239D4A","b":"5573744A7000","o":"2D92D4A","s":"_ZN5mongo15printStackTraceERSo","s+":"3A"}
Frame: {"a":"557377202D4D","b":"5573744A7000","o":"2D5BD4D","s":"_ZN5mongo5logv214FileRotateSink7consumeERKN5boost3log12v2s_mt_posix11record_viewERKNSt7__cxx1112basic_stringIcSt11char_traitsIcESaIcEEE","s+":"50D"}
Frame: {"a":"55737721A4E3","b":"5573744A7000","o":"2D734E3","s":"_ZN5boost3log12v2s_mt_posix5sinks13unlocked_sinkIN5mongo5logv216CompositeBackendIJNS5_14FileRotateSinkENS5_10RamLogSinkES8_NS5_14UserAssertSinkEEEEE7consumeERKNS1_11record_viewE","s+":"213"}
Frame: {"a":"55737734F223","b":"5573744A7000","o":"2EA8223","s":"_ZN5boost3log12v2s_mt_posix4core16push_record_moveERNS1_6recordE","s+":"1D3"}
Frame: {"a":"55737720E3E2","b":"5573744A7000","o":"2D673E2","s":"_ZN5mongo5logv26detail9doLogImplEiRKNS0_11LogSeverityERKNS0_10LogOptionsENS_10StringDataERKNS0_26TypeErasedAttributeStorageE","s+":"682"}
Frame: {"a":"557376BDD138","b":"5573744A7000","o":"2736138"plicationEPNS_14ServiceContextE","s+":"405"},{"a":"557375472FA2","b":"5573744A7000","o":"FCBFA2","s":"_ZN5mongo12_GLOBAL__N_111mongoDbMainEiPPcS2_","s+":"2C2"},{"a":"5573753E6029","b":"5573744A7000","o":"F3F029","s":"main","s+":"9"},{"a":"7EFCED2E33D5","b":"7EFCED2C1000","o":"223D5","s":"__libc_start_main","s+":"F5"},{"a":"55737546E94E","b":"5573744A7000","o":"FC794E","s":"_start","s+":"29"}],"processInfo":{"mongodbVersion":"4.4.6","gitVersion":"72e66213c2c3eab37d9358d5e78ad7f5c1d0d0d7","compiledModules":[],"uname":{"sysname":"Linux","release":"3.10.0-862.14.4.el7.x86_64","version":"#1 SMP Fri Sep 21 09:07:21 UTC 2018","machine":"x86_64"},"somap":[{"b":"5573744A7000","elfType":3,"buildId":"312311B4128B98F2F0502772FB85CC572CDDD0A1"},{"b":"7EFCED2C1000","path":"/lib64/libc.so.6","elfType":3,"buildId":"85EA0AE559B53AB60D8548242CEDD0E83F4816DA"}]}}
Frame: {"a":"55737723827A","b":"5573744A7000","o":"2D9127A","s":"_ZN5mongo18stack_trace_detail12_GLOBAL__N_119printStackTraceImplERKNS1_7OptionsEPNS_14StackTraceSinkUncaught exception
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "/data_local/workspace/conda_amine/lib/python3.7/site-packages/fiftyone/__init__.py", line 25, in <module>
from fiftyone.__public__ import *
File "/data_local/workspace/conda_amine/lib/python3.7/site-packages/fiftyone/__public__.py", line 11, in <module>
_database_service = fos.DatabaseService()
File "/data_local/workspace/conda_amine/lib/python3.7/site-packages/fiftyone/core/service.py", line 203, in __init__
super().__init__()
File "/data_local/workspace/conda_amine/lib/python3.7/site-packages/fiftyone/core/service.py", line 81, in __init__
self.start()
File "/data_local/workspace/conda_amine/lib/python3.7/site-packages/fiftyone/core/service.py", line 293, in start
food.set_default_port(self.port)
File "/data_local/workspace/conda_amine/lib/python3.7/site-packages/fiftyone/core/service.py", line 283, in port
return self._wait_for_child_port()
File "/data_local/workspace/conda_amine/lib/python3.7/site-packages/fiftyone/core/service.py", line 180, in _wait_for_child_port
return find_port()
File "/data_local/workspace/conda_amine/lib/python3.7/site-packages/retrying.py", line 49, in wrapped_f
return Retrying(*dargs, **dkw).call(f, *args, **kw)
File "/data_local/workspace/conda_amine/lib/python3.7/site-packages/retrying.py", line 212, in call
raise attempt.get()
File "/data_local/workspace/conda_amine/lib/python3.7/site-packages/retrying.py", line 247, in get
six.reraise(self.value[0], self.value[1], self.value[2])
File "/data_local/workspace/conda_amine/lib/python3.7/site-packages/six.py", line 719, in reraise
raise value
File "/data_local/workspace/conda_amine/lib/python3.7/site-packages/retrying.py", line 200, in call
attempt = Attempt(fn(*args, **kwargs), attempt_number, False)
File "/data_local/workspace/conda_amine/lib/python3.7/site-packages/fiftyone/core/service.py", line 178, in find_port
raise ServiceListenTimeout(etau.get_class_name(self), port)
fiftyone.core.service.ServiceListenTimeout: fiftyone.core.service.DatabaseService failed to bind to port
Hi @AmineDiro. Pardon the delay. I think what would be easiest until we can officially support RHEL (soon) is a quick patch that assumes you are running your own mongod
service.
All you should need to do is replace this line with
import fiftyone.core.odm.database as food
food.set_default_port(27017) # change to your mongo service port
food.get_db_conn()
Again, you should replace or comment out the line starts that initializes the DatabaseService
.
Hello thanks for the help again :smiley:
I would love to contribute to the support of RHEL (what would be the step to contribute on this ? )
i changes __public__
to :
11 #_database_service = fos.DatabaseService()
12 import fiftyone.core.odm.database as food
13 food.set_default_port(8889) # change to your mongo service port
14 food.get_db_conn()
Now everything is ok :
Python 3.7.10 (default, Jun 4 2021, 14:48:32)
[GCC 7.5.0] :: Anaconda, Inc. on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import fiftyone as fo
>>> dataset = fo.Dataset("test-dataset")
>>> print(dataset)
Name: test-dataset
Media type: None
Num samples: 0
Persistent: False
Tags: []
Sample fields:
id: fiftyone.core.fields.ObjectIdField
filepath: fiftyone.core.fields.StringField
tags: fiftyone.core.fields.ListField(fiftyone.core.fields.StringField)
metadata: fiftyone.core.fields.EmbeddedDocumentField(fiftyone.core.metadata.Metadata)
>>> print(fo.config)
{
"database_dir": "/users/l0494241/.fiftyone/var/lib/mongo",
"dataset_zoo_dir": "/users/l0494241/fiftyone",
"dataset_zoo_manifest_paths": null,
"default_app_port": 5151,
"default_batch_size": null,
"default_dataset_dir": "/users/l0494241/fiftyone",
"default_image_ext": ".jpg",
"default_ml_backend": "torch",
"default_sequence_idx": "%06d",
"default_video_ext": ".mp4",
"desktop_app": false,
"do_not_track": false,
"model_zoo_dir": "/users/l0494241/fiftyone/__models__",
"model_zoo_manifest_paths": null,
"requirement_error_level": 0,
"show_progress_bars": true
}
You'd need to add the download url option to the db package's setup.py
. And add a new rhel
package like debian9
.
https://github.com/voxel51/fiftyone/blob/develop/package/db/setup.py
In the end, the changes should allow for the following build command:
FIFTYONE_DB_BUILD_LINUX_DISTRO=rhel python setup.py bdist_wheel --plat-name linux-x86_64
I should note that I was able to install and use fiftyone
on a rhel 8 docker image, but I was not able to find a rhel 7 image. Perhaps I should check CentOS 7.
Adding arm wheel(s) would be nice too.
This page can help you generate the links, but remember we are using 4.4.2
Great Thanks for the guidance. I'll start working on this. :+1:
System information
fiftyone --version
): fiftyone v0.11.1, Voxel51, Inc.Commands to reproduce
pip install fiftyone
mongodb-org-server-4.4.6
system wide using dzdo account on RedHat because of import errorRuntimeError: Could not find mongod >= 4.4
python
import fiftyone as fo
Describe the problem
mongodb-org-server-4.4.6
system wide using dzdo account ont RH because of import errorRuntimeError: Could not find mongod >= 4.4
mongod --version db version v4.4.6 Build Info: { "version": "4.4.6", "gitVersion": "72e66213c2c3eab37d9358d5e78ad7f5c1d0d0d7", "openSSLVersion": "OpenSSL 1.0.1e-fips 11 Feb 2013", "modules": [], "allocator": "tcmalloc", "environment": { "distmod": "rhel70", "distarch": "x86_64", "target_arch": "x86_64" } }
fiftyone
in python i get the errorfiftyone.core.service.ServiceListenTimeout: fiftyone.core.service.DatabaseService failed to bind to port
Python 3.7.10 (default, Jun 4 2021, 14:48:32)
[GCC 7.5.0] :: Anaconda, Inc. on linux Type "help", "copyright", "credits" or "license" for more information.
When looking at similar issues i tried to run the mongod from the env but got and error : `[/data_local/workspace/conda_amine] l0494241@xrai-frpau-132 bin → /data_local/workspace/conda_amine/lib/python3.7/site-packages/fiftyone/db/bin/mongod --version
/data_local/workspace/conda_amine/lib/python3.7/site-packages/fiftyone/db/bin/mongod: error while loading shared libraries: libcrypto.so.1.1: cannot open shared object file: No such file or directory`
/usr/bin/mongod --dbpath /users/l0494241/.fiftyone/var/lib/mongo \ --logpath /users/l0494241/.fiftyone/var/lib/mongo/log/mongo.log \ --port 0 --nounixsocket
gives a core dump error :
.fiftyone
directory is created, this may be the issue, how can i configure fiftyone to use other directory for sertup ?Thanks for your help
Willingness to contribute
The FiftyOne Community encourages bug fix contributions. Would you or another member of your organization be willing to contribute a fix for this bug to the FiftyOne codebase?