gigablast / open-source-search-engine

Nov 20 2017 -- A distributed open source search engine and spider/crawler written in C/C++ for Linux on Intel/AMD. From gigablast dot com, which has binaries for download. See the README.md file at the very bottom of this page for instructions.
Apache License 2.0
1.54k stars 441 forks source link

Segmentation fault #199

Open k7n2g opened 1 year ago

k7n2g commented 1 year ago

ubuntu 18.02 node v16.18.1 30 GB RAM 800 GB SSD nothing in logs starts and runs fine then does a clean core dump tried various other node versions all have various problems in compiling compiles okay on 16.18.1 this is terminal output not much to go on sorry any help appreciated

1674167397804 000 db: gb is now ready 1674167397815 000 gb: clock is now synced with host #0. 1674167398207 000 thread: Using 36708352 bytes for 20 thread stacks. 1674167398525 000 spider: made spidercoll=5603bed6ab20 for cr=5603bab6c670 1674167398527 000 db: Loading waitingtree-saved.dat.

1674167398600 000 db: missed calling niceness 0 heartbeatWrapper function by 293 ms. Either you need a quick poll somewhere or a niceness 0 function is taking too long. Num elapsed alarms = 19 1674167398615 000 spider: rebuild complete for main. Added 1 recs to waiting tree, scanned 294 bytes of spiderdb. 1674167398814 000 loop: sigbadhandler. disabling handler from recall. 1674167398814 000 gb: Printing stack trace. use 'addr2line -e gb' to decode the hex below. 1674167398815 000 addr2line -e gb 0x5603b8fa94f1 1674167398815 000 addr2line -e gb 0x5603b8faa239 1674167398815 000 addr2line -e gb 0x7f7355ccc980 1674167398815 000 addr2line -e gb 0x5603b9074cab 1674167398815 000 addr2line -e gb 0x5603b9075556 1674167398815 000 addr2line -e gb 0x5603b9075afa 1674167398815 000 addr2line -e gb 0x5603b9075d05 1674167398815 000 addr2line -e gb 0x5603b8f2b1e9 1674167398816 000 addr2line -e gb 0x5603b8f2b8f2 1674167398816 000 addr2line -e gb 0x5603b8f2bdda 1674167398816 000 addr2line -e gb 0x5603b8fa8c4f 1674167398816 000 addr2line -e gb 0x5603b8fa997a 1674167398816 000 addr2line -e gb 0x5603b8fa9f0f 1674167398816 000 addr2line -e gb 0x5603b8e5ce2e 1674167398816 000 addr2line -e gb 0x5603b8e562da 1674167398816 000 addr2line -e gb 0x7f73545b2c87 1674167398816 000 addr2line -e gb 0x5603b8e5614a 1674167398816 000 loop: sigbadhandler. trying to save now. mode=0 1674167398816 000 mem: checking mem for breeches 1674167399028 000 gb: Shutting down urgently. Timed try #0. 1674167399028 000 gb: disabling threads 1674167399029 000 gb: trying to shutdown 1674167399029 000 gb: calling save tree for titledb 1674167399029 000 gb: calling save tree for sectiondb 1674167399029 000 gb: calling save tree for posdb 1674167399029 000 gb: calling save tree for spiderdb 1674167399029 000 gb: calling save tree for clusterdb 1674167399029 000 gb: calling save tree for tagdb 1674167399029 000 gb: calling save tree for catdb 1674167399029 000 gb: calling save tree for statsdb 1674167399029 000 gb: calling save tree for linkdb 1674167399029 000 gb: calling save tree for rdbid 0 1674167399029 000 gb: calling save tree for rdbid 0 1674167399029 000 gb: calling save tree for rdbid 0 1674167399029 000 gb: calling save tree for doledb 1674167399029 000 db: Saving /home/seed8/open-source-search-engine//doledb-saved.dat 1674167399030 000 gb: calling save tree for rdbid 0 1674167399030 000 gb: calling save tree for rdbid 0 1674167399030 000 gb: calling save tree for rdbid 0 1674167399030 000 gb: calling save tree for rdbid 0 1674167399030 000 gb: calling save tree for rdbid 0 1674167399030 000 gb: calling save tree for rdbid 0 1674167399030 000 gb: calling save tree for rdbid 0 1674167399030 000 spider: saving waiting tree for cn=0 1674167399030 000 db: Saving /home/seed8/open-source-search-engine/coll.main.0/waitingtree-saved.dat 1674167399030 000 gb: Shutting down udp server port 5998. 1674167399030 000 gb: Closing udp server socket port 5998. 1674167399030 000 gb: Shut down udp server port 5998 successfully. 1674167399030 000 gb: Shutting down udp server port 9000. 1674167399030 000 gb: Closing udp server socket port 9000. 1674167399030 000 gb: Shut down udp server port 9000 successfully. 1674167399030 000 admin: stopping real time profiler 1674167399030 000 gb: Saving miscellaneous data files. 1674167399032 000 build: Saving /home/seed8/open-source-search-engine/addsinprogress.saving 1674167399032 000 build: Renamed /home/seed8/open-source-search-engine/addsinprogress.saving to /home/seed8/open-source-search-engine/addsinprogress.dat 1674167399032 000 db: Saving 128000 bytes of cache to /home/seed8/open-source-search-engine//dns.cache 1674167399034 000 gb: calling abort from main process with pid of 22677 (main process) 1674167399034 000 gb: Joining with all threads 1674167399034 000 threads: killing all threads 1674167399034 000 gb: Dumping core after saving. Segmentation fault (core dumped)

k7n2g commented 1 year ago

Sorry haven't closed this i sorted out the problem now running thanks

tcreek commented 1 year ago

What was the problem?

k7n2g commented 1 year ago

Missing files

tcreek commented 1 year ago

I have not tried running this in over a year. I saw there was a merged pull request from about 8 months ago. Reason I had not been running, before I was getting an error.

Today, I have gotten further in running it. It runs until I try adding a URL for spider crawl. I then get the same error as you.

What files were missing?

k7n2g commented 1 year ago

Hi tcreek list the files you have and I will try and send you those I have that you are missing if you still need them

tcreek commented 1 year ago

Hi tcreek list the files you have and I will try and send you those I have that you are missing if you still need them

I have the files per the clone of the Master Branch, so I am not sure what you would be looking for, or what would even be missing.

k7n2g commented 1 year ago

sorry it's an email to someone else that got put in your box

tcreek commented 1 year ago

Still no idea what you are referring to

tcreek commented 6 months ago

Here is the answer

https://github.com/gigablast/open-source-search-engine/issues/204#issuecomment-2050056457