search

xbapps / xbvr

Tool to organize and stream your VR porn library
338 stars 130 forks source link

Matched scenes play only trailer, unmatched scenes play fine #1663

Closed metacarejeff closed 7 months ago

metacarejeff commented 7 months ago

Scenes files are listed and play properly in webui. Matched scenes are listed in Heresphere&Deovr, but with trailer length Unmatched scenes play properly in deovr & heresphere

If I match a scene that I just played in deovr / heresphere, when I go in and play that scene now that it is matched, it plays only the trailer.

I have no idea.

2024/03/18 14:29:50 http: panic serving [fe80::aaea:4df9:bd6b:ad66%enp45s0]:63115: runtime error: index out of range [0] with length 0 [dev:go] goroutine 3478 [running]: [dev:go] net/http.(conn).serve.func1() [dev:go] /usr/local/go/src/net/http/server.go:1898 +0xbe [dev:go] panic({0x14b3200?, 0xc005718f18?}) [dev:go] /usr/local/go/src/runtime/panic.go:770 +0x132 [dev:go] github.com/xbapps/xbvr/pkg/api.DeoVRResource.getDeoScene({}, 0xc0027907e0, 0xc000418070) [dev:go] /home/jeff/xbvr/pkg/api/deovr.go:464 +0x2edc [dev:go] github.com/emicklei/go-restful/v3.(FilterChain).ProcessFilter(0x7ff6600b2f18?, 0x411efa?, 0x600008?) [dev:go] /home/jeff/go/pkg/mod/github.com/emicklei/go-restful/v3@v3.11.2/filter.go:23 +0x53 [dev:go] github.com/xbapps/xbvr/pkg/api.restfulAuthFilter(0x50?, 0x14a44e0?, 0x1?) [dev:go] /home/jeff/xbvr/pkg/api/deovr.go:226 +0x20f [dev:go] github.com/emicklei/go-restful/v3.(FilterChain).ProcessFilter(0xc0002d2708?, 0x2668150?, 0xc0027e57d0?) [dev:go] /home/jeff/go/pkg/mod/github.com/emicklei/go-restful/v3@v3.11.2/filter.go:21 +0x42 [dev:go] github.com/emicklei/go-restful/v3.(Container).dispatch(0xc000194d80, {0x2667ca0, 0xc00090fce0}, 0xc000a52d80) [dev:go] /home/jeff/go/pkg/mod/github.com/emicklei/go-restful/v3@v3.11.2/container.go:296 +0x9e5 [dev:go] net/http.HandlerFunc.ServeHTTP(0x3207230?, {0x2667ca0?, 0xc00090fce0?}, 0x152f880?) [dev:go] /usr/local/go/src/net/http/server.go:2166 +0x29 [dev:go] net/http.(ServeMux).ServeHTTP(0xc000a52c60?, {0x2667ca0, 0xc00090fce0}, 0xc000a52d80) [dev:go] /usr/local/go/src/net/http/server.go:2683 +0x1ad [dev:go] github.com/gorilla/mux.(Router).ServeHTTP(0xc0001d8a80, {0x2667ca0, 0xc00090fce0}, 0xc000ab8120) [dev:go] /home/jeff/go/pkg/mod/github.com/gorilla/mux@v1.8.1/mux.go:212 +0x1e2 [dev:go] github.com/xbapps/xbvr/pkg/server.StartServer.(Cors).Handler.func9({0x2667ca0, 0xc00090fce0}, 0xc000ab8120) [dev:go] /home/jeff/go/pkg/mod/github.com/rs/cors@v1.10.1/cors.go:281 +0x184 [dev:go] net/http.HandlerFunc.ServeHTTP(0x475199?, {0x2667ca0?, 0xc00090fce0?}, 0xc0009cfb68?) [dev:go] /usr/local/go/src/net/http/server.go:2166 +0x29 [dev:go] net/http.serverHandler.ServeHTTP({0xc005688090?}, {0x2667ca0?, 0xc00090fce0?}, 0x6?) /usr/local/go/src/net/http/server.go:3137 +0x8e [dev:go] net/http.(conn).serve(0xc00029e090, {0x266bd48, 0xc000880210}) [dev:go] /usr/local/go/src/net/http/server.go:2039 +0x5e8 [dev:go] created by net/http.(Server).Serve in goroutine 1 [dev:go] /usr/local/go/src/net/http/server.go:3285 +0x4b4 [dev:go] 2024/03/18 14:29:51 http: panic serving [fe80::aaea:4df9:bd6b:ad66%enp45s0]:63116: runtime error: index out of range [0] with length 0 [dev:go] goroutine 3815 [running]: [dev:go] net/http.(conn).serve.func1() [dev:go] /usr/local/go/src/net/http/server.go:1898 +0xbe [dev:go] panic({0x14b3200?, 0xc006f087c8?}) [dev:go] /usr/local/go/src/runtime/panic.go:770 +0x132 [dev:go] github.com/xbapps/xbvr/pkg/api.DeoVRResource.getDeoScene({}, 0xc006f20040, 0xc0004240e0) [dev:go] /home/jeff/xbvr/pkg/api/deovr.go:464 +0x2edc [dev:go] github.com/emicklei/go-restful/v3.(FilterChain).ProcessFilter(0xc00022b688?, 0x422f36?, 0x33f7e20?) [dev:go] /home/jeff/go/pkg/mod/github.com/emicklei/go-restful/v3@v3.11.2/filter.go:23 +0x53 [dev:go] github.com/xbapps/xbvr/pkg/api.restfulAuthFilter(0x50?, 0x14a44e0?, 0x1?) [dev:go] /home/jeff/xbvr/pkg/api/deovr.go:226 +0x20f [dev:go] github.com/emicklei/go-restful/v3.(FilterChain).ProcessFilter(0xc000152e08?, 0x2668150?, 0xc006f02240?) [dev:go] /home/jeff/go/pkg/mod/github.com/emicklei/go-restful/v3@v3.11.2/filter.go:21 +0x42 [dev:go] github.com/emicklei/go-restful/v3.(Container).dispatch(0xc000194d80, {0x2667ca0, 0xc00090fb20}, 0xc00082e480) [dev:go] /home/jeff/go/pkg/mod/github.com/emicklei/go-restful/v3@v3.11.2/container.go:296 +0x9e5 [dev:go] net/http.HandlerFunc.ServeHTTP(0x3207230?, {0x2667ca0?, 0xc00090fb20?}, 0x152f880?) [dev:go] /usr/local/go/src/net/http/server.go:2166 +0x29 [dev:go] net/http.(ServeMux).ServeHTTP(0xc00082e360?, {0x2667ca0, 0xc00090fb20}, 0xc00082e480) /usr/local/go/src/net/http/server.go:2683 +0x1ad [dev:go] github.com/gorilla/mux.(Router).ServeHTTP(0xc0001d8a80, {0x2667ca0, 0xc00090fb20}, 0xc00082f200) [dev:go] /home/jeff/go/pkg/mod/github.com/gorilla/mux@v1.8.1/mux.go:212 +0x1e2 [dev:go] github.com/xbapps/xbvr/pkg/server.StartServer.(Cors).Handler.func9({0x2667ca0, 0xc00090fb20}, 0xc00082f200) [dev:go] /home/jeff/go/pkg/mod/github.com/rs/cors@v1.10.1/cors.go:281 +0x184 [dev:go] net/http.HandlerFunc.ServeHTTP(0x475199?, {0x2667ca0?, 0xc00090fb20?}, 0xc0009cfb68?) [dev:go] /usr/local/go/src/net/http/server.go:2166 +0x29 [dev:go] net/http.serverHandler.ServeHTTP({0xc002a46e10?}, {0x2667ca0?, 0xc00090fb20?}, 0x6?) [dev:go] /usr/local/go/src/net/http/server.go:3137 +0x8e [dev:go] net/http.(conn).serve(0xc000b37b00, {0x266bd48, 0xc000880210}) [dev:go] /usr/local/go/src/net/http/server.go:2039 +0x5e8 [dev:go] created by net/http.(Server).Serve in goroutine 1 [dev:go] /usr/local/go/src/net/http/server.go:3285 +0x4b4 [dev:go] 2024/03/18 14:29:51 http: panic serving [fe80::aaea:4df9:bd6b:ad66%enp45s0]:63117: runtime error: index out of range [0] with length 0 [dev:go] goroutine 3816 [running]: [dev:go] net/http.(conn).serve.func1() [dev:go] /usr/local/go/src/net/http/server.go:1898 +0xbe [dev:go] panic({0x14b3200?, 0xc006f08f48?}) [dev:go] /usr/local/go/src/runtime/panic.go:770 +0x132 [dev:go] github.com/xbapps/xbvr/pkg/api.DeoVRResource.getDeoScene({}, 0xc006f207c0, 0xc000424230) [dev:go] /home/jeff/xbvr/pkg/api/deovr.go:464 +0x2edc [dev:go] github.com/emicklei/go-restful/v3.(FilterChain).ProcessFilter(0x7ff6600b2a68?, 0x411efa?, 0x780008?) [dev:go] /home/jeff/go/pkg/mod/github.com/emicklei/go-restful/v3@v3.11.2/filter.go:23 +0x53 [dev:go] github.com/xbapps/xbvr/pkg/api.restfulAuthFilter(0x50?, 0x14a44e0?, 0x1?) /home/jeff/xbvr/pkg/api/deovr.go:226 +0x20f [dev:go] github.com/emicklei/go-restful/v3.(FilterChain).ProcessFilter(0xc000c31888?, 0x2668150?, 0xc006f5f8c0?) [dev:go] /home/jeff/go/pkg/mod/github.com/emicklei/go-restful/v3@v3.11.2/filter.go:21 +0x42 [dev:go] github.com/emicklei/go-restful/v3.(Container).dispatch(0xc000194d80, {0x2667ca0, 0xc00090fce0}, 0xc000a52fc0) [dev:go] /home/jeff/go/pkg/mod/github.com/emicklei/go-restful/v3@v3.11.2/container.go:296 +0x9e5 [dev:go] net/http.HandlerFunc.ServeHTTP(0x3207230?, {0x2667ca0?, 0xc00090fce0?}, 0x152f880?) [dev:go] /usr/local/go/src/net/http/server.go:2166 +0x29 [dev:go] net/http.(ServeMux).ServeHTTP(0xc000a52ea0?, {0x2667ca0, 0xc00090fce0}, 0xc000a52fc0) [dev:go] /usr/local/go/src/net/http/server.go:2683 +0x1ad [dev:go] github.com/gorilla/mux.(Router).ServeHTTP(0xc0001d8a80, {0x2667ca0, 0xc00090fce0}, 0xc00082f320) [dev:go] /home/jeff/go/pkg/mod/github.com/gorilla/mux@v1.8.1/mux.go:212 +0x1e2 [dev:go] github.com/xbapps/xbvr/pkg/server.StartServer.(Cors).Handler.func9({0x2667ca0, 0xc00090fce0}, 0xc00082f320) [dev:go] /home/jeff/go/pkg/mod/github.com/rs/cors@v1.10.1/cors.go:281 +0x184 [dev:go] net/http.HandlerFunc.ServeHTTP(0x475199?, {0x2667ca0?, 0xc00090fce0?}, 0xc00094ab68?) [dev:go] /usr/local/go/src/net/http/server.go:2166 +0x29 [dev:go] net/http.serverHandler.ServeHTTP({0xc002a46ed0?}, {0x2667ca0?, 0xc00090fce0?}, 0x6?) [dev:go] /usr/local/go/src/net/http/server.go:3137 +0x8e [dev:go] net/http.(conn).serve(0xc000b37b90, {0x266bd48, 0xc000880210}) [dev:go] /usr/local/go/src/net/http/server.go:2039 +0x5e8 [dev:go] created by net/http.(Server).Serve in goroutine 1 [dev:go] /usr/local/go/src/net/http/server.go:3285 +0x4b4 [dev:go] 2024/03/18 14:29:51 http: panic serving [fe80::aaea:4df9:bd6b:ad66%enp45s0]:63118: runtime error: index out of range [0] with length 0 [dev:go] goroutine 3845 [running]: [dev:go] net/http.(conn).serve.func1() [dev:go] /usr/local/go/src/net/http/server.go:1898 +0xbe [dev:go] panic({0x14b3200?, 0xc006f096c8?}) [dev:go] /usr/local/go/src/runtime/panic.go:770 +0x132 [dev:go] github.com/emicklei/go-restful/v3.(Container).dispatch(0xc000194d80, {0x2667ca0, 0xc00090fea0}, 0xc0004a9d40) [dev:go] /home/jeff/go/pkg/mod/github.com/emicklei/go-restful/v3@v3.11.2/container.go:296 +0x9e5 [dev:go] net/http.HandlerFunc.ServeHTTP(0x3207230?, {0x2667ca0?, 0xc00090fea0?}, 0x152f880?) /usr/local/go/src/net/http/server.go:2166 +0x29github.com/xbapps/xbvr/pkg/api.DeoVRResource.getDeoScene({}, 0xc006f20f60, 0xc0004243f0) [dev:go] /home/jeff/xbvr/pkg/api/deovr.go:464 +0x2edc [dev:go] github.com/emicklei/go-restful/v3.(FilterChain).ProcessFilter(0x7ff6600b2a68?, 0x411efa?, 0x780008?) [dev:go] /home/jeff/go/pkg/mod/github.com/emicklei/go-restful/v3@v3.11.2/filter.go:23 +0x53 [dev:go] github.com/xbapps/xbvr/pkg/api.restfulAuthFilter(0x50?, 0x14a44e0?, 0x1?) [dev:go] /home/jeff/xbvr/pkg/api/deovr.go:226 +0x20f [dev:go] github.com/emicklei/go-restful/v3.(FilterChain).ProcessFilter(0xc000936008?, 0x2668150?, 0xc007099080?) [dev:go] /home/jeff/go/pkg/mod/github.com/emicklei/go-restful/v3@v3.11.2/filter.go:21 +0x42 [dev:go] [dev:go] net/http.(ServeMux).ServeHTTP(0xc0004a9c20?, {0x2667ca0, 0xc00090fea0}, 0xc0004a9d40) [dev:go] /usr/local/go/src/net/http/server.go:2683 +0x1ad [dev:go] github.com/gorilla/mux.(Router).ServeHTTP(0xc0001d8a80, {0x2667ca0, 0xc00090fea0}, 0xc0004a98c0) [dev:go] /home/jeff/go/pkg/mod/github.com/gorilla/mux@v1.8.1/mux.go:212 +0x1e2 [dev:go] github.com/xbapps/xbvr/pkg/server.StartServer.(Cors).Handler.func9({0x2667ca0, 0xc00090fea0}, 0xc0004a98c0) [dev:go] /home/jeff/go/pkg/mod/github.com/rs/cors@v1.10.1/cors.go:281 +0x184 [dev:go] net/http.HandlerFunc.ServeHTTP(0x475199?, {0x2667ca0?, 0xc00090fea0?}, 0xc00094ab68?) [dev:go] /usr/local/go/src/net/http/server.go:2166 +0x29 [dev:go] net/http.serverHandler.ServeHTTP({0xc007098f00?}, {0x2667ca0?, 0xc00090fea0?}, 0x6?) [dev:go] /usr/local/go/src/net/http/server.go:3137 +0x8e net/http.(conn).serve(0xc000b37cb0, {0x266bd48, 0xc000880210}) [dev:go] /usr/local/go/src/net/http/server.go:2039 +0x5e8 [dev:go] created by net/http.(Server).Serve in goroutine 1 [dev:go] /usr/local/go/src/net/http/server.go:3285 +0x4b4 [dev:go] time="2024-03-18T14:29:52Z" level=info msg="Adding scraped scenes to search index..." task=scrape [dev:go] time="2024-03-18T14:29:52Z" level=info msg="Indexed 2 scenes" task=scrape [dev:go] time="2024-03-18T14:29:52Z" level=info msg="Scraped 2 new scenes in 4s" task=scrape [dev:go] 2024/03/18 14:29:56 http: panic serving [fe80::aaea:4df9:bd6b:ad66%enp45s0]:63120: runtime error: index out of range [0] with length 0 [dev:go] goroutine 3954 [running]: [dev:go] net/http.(conn).serve.func1() [dev:go] /usr/local/go/src/net/http/server.go:1898 +0xbe [dev:go] panic({0x14b3200?, 0xc002218e70?}) [dev:go] /usr/local/go/src/runtime/panic.go:770 +0x132 [dev:go] github.com/xbapps/xbvr/pkg/api.DeoVRResource.getDeoScene({}, 0xc00103e180, 0xc0009a02a0) [dev:go] /home/jeff/xbvr/pkg/api/deovr.go:464 +0x2edc [dev:go] github.com/emicklei/go-restful/v3.(FilterChain).ProcessFilter(0x7ff6600b4b38?, 0x411efa?, 0x801808?) [dev:go] /home/jeff/go/pkg/mod/github.com/emicklei/go-restful/v3@v3.11.2/filter.go:23 +0x53 [dev:go] github.com/xbapps/xbvr/pkg/api.restfulAuthFilter(0x50?, 0x14a44e0?, 0x1?) [dev:go] /home/jeff/xbvr/pkg/api/deovr.go:226 +0x20f [dev:go] github.com/emicklei/go-restful/v3.(FilterChain).ProcessFilter(0xc0002d2388?, 0x2668150?, 0xc001e0f950?) [dev:go] /home/jeff/go/pkg/mod/github.com/emicklei/go-restful/v3@v3.11.2/filter.go:21 +0x42 [dev:go] github.com/emicklei/go-restful/v3.(Container).dispatch(0xc000194d80, {0x2667ca0, 0xc001090460}, 0xc00120d200) [dev:go] /home/jeff/go/pkg/mod/github.com/emicklei/go-restful/v3@v3.11.2/container.go:296 +0x9e5 [dev:go] net/http.HandlerFunc.ServeHTTP(0x3207230?, {0x2667ca0?, 0xc001090460?}, 0x152f880?) [dev:go] /usr/local/go/src/net/http/server.go:2166 +0x29 [dev:go] net/http.(ServeMux).ServeHTTP(0xc00120d0e0?, {0x2667ca0, 0xc001090460}, 0xc00120d200) [dev:go] /usr/local/go/src/net/http/server.go:2683 +0x1ad [dev:go] github.com/gorilla/mux.(Router).ServeHTTP(0xc0001d8a80, {0x2667ca0, 0xc001090460}, 0xc00120cfc0) [dev:go] /home/jeff/go/pkg/mod/github.com/gorilla/mux@v1.8.1/mux.go:212 +0x1e2 [dev:go] github.com/xbapps/xbvr/pkg/server.StartServer.(Cors).Handler.func9({0x2667ca0, 0xc001090460}, 0xc00120cfc0) [dev:go] /home/jeff/go/pkg/mod/github.com/rs/cors@v1.10.1/cors.go:281 +0x184 [dev:go] net/http.HandlerFunc.ServeHTTP(0x475199?, {0x2667ca0?, 0xc001090460?}, 0xc0000a9b68?) [dev:go] /usr/local/go/src/net/http/server.go:2166 +0x29 [dev:go] net/http.serverHandler.ServeHTTP({0xc001e0f7d0?}, {0x2667ca0?, 0xc001090460?}, 0x6?) [dev:go] /usr/local/go/src/net/http/server.go:3137 +0x8e [dev:go] net/http.(conn).serve(0xc01a79e750, {0x266bd48, 0xc000880210}) [dev:go] /usr/local/go/src/net/http/server.go:2039 +0x5e8 [dev:go] created by net/http.(*Server).Serve in goroutine 1 [dev:go] /usr/local/go/src/net/http/server.go:3285 +0x4b4

theRealKLH commented 7 months ago

your cache might be corrupted. I would go to cache settings and reset/rescan index/reset scenes.

metacarejeff commented 7 months ago

I found the cause of the issue, but the resolution I had to go with was to drop the DB and start over.
The issue happened because I linked my StashDB to XBVR. Either the scrape or display code was over-riding the files with the links to the stash (Both had the same files on them), and the extern links.

Like I said, I know that something in that interaction caused it, but I saw no way to unwind it where it started working. I tried to unlink scenes, but that did not work (No errors in log either). I still have the old DB, and I can attach it.

My plan was to use stash + fingerprint for extra scraping ability, and the name standardization. I decided to just write sql scripts to rename at this point.

So I now have scripts that: delete duplicate scenes (smallest file), rename scenes, Move unmatched scenes to a folder (Because there is no folder filter/sort inside "Files", concatenate multipart scenes (JAV files).

It seems like those sql queries were relatively simple to dump the command line for each, not sure why they are not part of the build.

Also, made a scraper for stashDB that does not try and LINK externally to stashDB, and just uses it to help id scenes and cross link them to the right studio/scene from other scrapers. It seemed like the least destructive path.

metacarejeff commented 7 months ago

Sorry I mention scripts and don't share, here: Bash script to bulk import scenes from list using csv Change addr to your xbvr, and the .csv file at the end to your file. it works with javland, javlibrary, and javdb or javdatabase whichever it is. while IFS= read -r line; do echo "$line" curl 'http://10.10.99.197:9111/api/task/scrape-javr' \ -H 'Accept: /' \ -H 'Accept-Language: en-US,en;q=0.9' \ -H 'Connection: keep-alive' \ -H 'Origin: http://10.10.99.197:9111' \ -H 'Referer: http://10.10.99.197:9111/ui/' \ -H 'User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/122.0.0.0 Safari/537.36' \ -H 'content-type: application/json' \ --data-raw '{"s":"javland","q":"'$line'"}' \ --insecure sleep 600 done <javdb-list2.csv

sqlite script to add filenames to scenes since none of the jav scrapers do it (I just use scene_id.mp4 in my lib) update scenes set filenames_arr = concat('["',scene_id,'-1.mp4","', scene_id,'-2.mp4","', scene_id,'-3.mp4","', scene_id,'-4.mp4","', scene_id,'-5.mp4","', scene_id,'-6.mp4","', scene_id,'-7.mp4","', scene_id,'-8.mp4","', scene_id,'-9.mp4","', scene_id,'-10.mp4","', scene_id,'-11.mp4","', scene_id,'-12.mp4","', scene_id,'-13.mp4","', scene_id,'-A.mp4","', scene_id,'-B.mp4","', scene_id,'-C.mp4","', scene_id,'-D.mp4","', scene_id,'-E.mp4","', scene_id,'-F.mp4","', scene_id,'-G.mp4","', scene_id,'-H.mp4","', scene_id,'-I.mp4","', scene_id,'-J.mp4","', scene_id,'-K.mp4","', scene_id,'-L.mp4","', scene_id,'-M.mp4","', scene_id,'-N.mp4","', scene_id,'-14.mp4"]' ) Where scraper_id =""

Script to dump list of concatenation commands for all of the scenes in the where clause SELECT concat('ffmpeg -f concat -safe 0 -i <(for f in ./',scene_id,'-*.mp4; do echo "file ''$PWD/$f''"; done) -c copy ',scene_id,'.mp4') from scenes where scraper_id ="" order by scene_id

Move all unmatched scenes command generation select concat('mv ',path,'/',filename, ' /dest/dir/', filename) from files where scene_id = 0

And this one finds duplicates lists both with size, and the command to delete them SELECT concat('rm ',path,'/',filename),size FROM files where scene_id in (select scene_id from files GROUP BY scene_id HAVING COUNT(*) > 1) and scene_id !=0 order by scene_id, size asc

If you go with top(1) or whatever, then you only get the smaller one, but I like to be able to check by hand, obviously throwing this into a web view could have more params, as with any of the other scripts.

theRealKLH commented 7 months ago

you should join the discord. might have saved yourself some scripting (unless of course you just like/love scripting). There's at least one person that has some scripting going on.

and.... glad you found your issue.

metacarejeff commented 7 months ago

I found the issue, and I just rebuilt the DB. I just exported and moved to mysql, and when i reimported it is the same behavior.

WEBUI sees the files, but heresphere plays only trailers. I think its something to do with external links but I cannot find it in the code to rip it out or figure out what to delete from config or whatever to make it go away!

Anyone?

metacarejeff commented 7 months ago

you should join the discord. might have saved yourself some scripting (unless of course you just like/love scripting). There's at least one person that has some scripting going on.

and.... glad you found your issue.

You were right, I just didn't quite understand. I rebuilt the index, but then I needed to rescan availability, that rectified it.

Thanks!