Ellerhold / fs2es-indexer

This tool indexes your directories into an elastic search index and prepares them for searching via Mac OS Spotlight search in a samba file server.
43 stars 3 forks source link

Finder not showing results. #13

Closed alexmf23 closed 2 years ago

alexmf23 commented 2 years ago

Hey, I am using Elasticsearch 8.1.2 with the fs3es-crawler on Ubuntu 20.4 for indexing SMB Shares. I am able to run the indexer and daemon, and i am also able to search with fs2es-indexer search --search-path /srv/samba --search-term

But when i try to search in Finder for a file on the share, it displays 0 results, while SMB logs shows that the files are found. Do you have maybe an idea about this ? Thanks.

MatthiasKuehneEllerhold commented 2 years ago

Today I've added some How-To and code in order to make fs2es-indexer work with ESv8. Could you please look into the README.md and tell me if this is helpful?

alexmf23 commented 2 years ago

I have looked over your update and installed it. I disabled elasticsearch security ( set xpack.security.enabled: false ) for testing without https protocol. Installation runs without problems, my share was successfully indexed and I can access http://localhost:9200

{
  "name" : "node-1",
  "cluster_name" : "ppt4u-cluster",
  "cluster_uuid" : "absXTdHoSZyt7yLg2dkzkw",
  "version" : {
    "number" : "8.1.2",
    "build_flavor" : "default",
    "build_type" : "deb",
    "build_hash" : "31df9689e80bad366ac20176aa7f2371ea5eb4c1",
    "build_date" : "2022-03-29T21:18:59.991429448Z",
    "build_snapshot" : false,
    "lucene_version" : "9.0.0",
    "minimum_wire_compatibility_version" : "7.17.0",
    "minimum_index_compatibility_version" : "7.0.0"
  },
  "tagline" : "You Know, for Search"
}

If i go on http://localhost:9200/files/_search it shows me files from inside my share. The problem comes when i try to search in Finder, it gives me 0 results, while smb_audit log shows files maching my search.

This is smb log, after i press search in Finder - but Finder return 0 items.

Screenshot 2022-04-13 at 15 48 50

Can this thing happen because im using Samba Version 4.13.17-Ubuntu ?

MatthiasKuehneEllerhold commented 2 years ago

Support for Elasticsearch was added in Samba 4.12, so 4.13 should work!

Did you configure the ES-details correctly in the smb.conf?

What packages are you using - are they self-built without spotlight support?

MatthiasKuehneEllerhold commented 2 years ago

So our tests just concluded: Debian 11 Bullseye with ES v8.1.2 with a user for fs2es-indexer and anonymous ro access to the index work. Our main problem was that data we migrated from a MacOS server had weird metadata in it. Which lead to a crash of the Mac OS finder and a mandatory restart of the Mac.

In order to fix this we need to

find . -type f -name ".DS_Store" -delete
find . -exec setfattr -x user.DOSATTRIB {} \;

And add these lines to our smb.conf

    veto files = /.DS_Store/
    delete veto files = yes
MatthiasKuehneEllerhold commented 2 years ago

Ok next step for your debugging: does mdfind in Mac find somethings?

First: mount the samba share

mdfind -onlyin /Volumes/my-volume/ my-search-term

If this command does not produce anything - wait 5 secs and try again!

MatthiasKuehneEllerhold commented 2 years ago

Ive added a whole section on how to debug to the README.md . Please let me know if this is helpful for you!

alexmf23 commented 2 years ago

Ive added a whole section on how to debug to the README.md . Please let me know if this is helpful for you!

I have rechecked everything and i found out that mdfind doesnt find any files and i cannot figure it out why. I've recked log files and it throwing me this:

Screenshot 2022-05-04 at 12 30 31
MatthiasKuehneEllerhold commented 2 years ago

Whats your Mac OS version? Whats your SMB Protocol Connection version (run smbutil statshares -a on your Mac)?

MatthiasKuehneEllerhold commented 2 years ago

Your log shows "HTTP connect failed", seems like there is a problem in your ES instance?!

alexmf23 commented 2 years ago

Your log shows "HTTP connect failed", seems like there is a problem in your ES instance?!

Let me recap: Server Samba Version: 4.13.17-Ubuntu Server ES Version: 7.17.5 Mac OS Version: 12.4 ( Monterey ) MAC SMB Protocol Connection: SMB_3 & SMB_2.1 ( i tried with both )

I fixed the 'HTTP connect failed' but still i get no results in finder window. The only error that samba throws now is this one:

[2022/06/29 13:01:17.806563,  1] ../../source3/smbd/smb2_server.c:2593(smbd_smb2_request_verify_creditcharge)
  smbd_smb2_request_verify_creditcharge: CreditCharge too low, given 1, needed 2

any ideas ?

MatthiasKuehneEllerhold commented 2 years ago

Sorry, I can't help you there. Try asking the samba mailing list for problem between Samba and ES. Seems like it is not a problem with the indexer.

If you manage to fix it, please post the solution here so I can add it to the README.