Open janaxhell opened 1 day ago
Hey Jana,
The path on your machine for the second volume in streamsphere and the volume in content require to be the same! I have made the fix in the docker compose below. Please try it out and let me know if this works for you? :-)
for the downloaded media, based on how yt-dlp works, it is likely that the downloaded videos should have been in /app/cmd which was not mounted to a permanent volume and should have been deleted when the running streamsphere container stopped.
I would also suggest that you remove the existing docker volumes and, containers and start afresh!
Thanks for providing the detailed logs and apologies for the late reply!
services:
streamsphere:
image: streamsphere/streamsphere:latest
# image: streamsphere/streamsphere:latest-arm
container_name: streamsphere
restart: unless-stopped
ports:
- 1282:3000
environment:
HOST_IP: "192.168.1.94"
APPLICATION_PORT: "1282"
CONTENT_PORT: "1288"
volumes:
- /srv/dev-disk-by-uuid-5b67514d-485e-4306-873e-b1cbb54ccf99/Config/streamsphere/db-data:/app/database/db
- /srv/dev-disk-by-uuid-BAF04088F0404D37/data/media/streamsphere:/app/content
content:
image: streamsphere/streamsphere-content:latest
# image: streamsphere/streamsphere-content:latest-arm
container_name: streamsphere-content
restart: unless-stopped
ports:
- 1288:3500
volumes:
- /srv/dev-disk-by-uuid-BAF04088F0404D37/data/media/streamsphere:/content
networks:
default:
external:
name: omv1
Hi, I've changed the binds to the same folder like this
volumes:
- /srv/dev-disk-by-uuid-5b67514d-485e-4306-873e-b1cbb54ccf99/Config/streamsphere/db-data:/app/database/db
- /srv/dev-disk-by-uuid-BAF04088F0404D37/data/media/streamsphere/content:/app/content
content:
image: streamsphere/streamsphere-content:latest
# image: streamsphere/streamsphere-content:latest-arm
container_name: streamsphere-content
restart: unless-stopped
ports:
- 1288:3500
volumes:
- /srv/dev-disk-by-uuid-BAF04088F0404D37/data/media/streamsphere/content:/content
but now the only thing that has changed is that I see the thumbnails while downloading (I didn't see any previously), but all that physically appears in the folder are the thumbnails. The app clearly shows the megabytes of the video downloading, but only a thumb is stored. Also, with previous compose thumbnails ended in the /config folder, now they are in the /content folder
? Fiber v2.52.5 ?
? http://127.0.0.1:3000 ?
? (bound on host 0.0.0.0 and port 3000) ?
? ?
? Handlers ............ 35 Processes ........... 1 ?
? Prefork ....... Disabled PID ................ 11 ?
?????????????????????????????????????????????????????
08:30:27 | 200 | 120.341µs | 192.168.1.21 | GET | / | -
2024/11/24 08:30:28.410752 content.go:179: [Info] Request Params:#0000000100000002 - 172.16.0.4:3000 <-> 192.168.1.21:54069 - GET http://192.168.1.94:1282/api/search/info
08:30:28 | 200 | 284.765µs | 192.168.1.21 | GET | /api/search/info | -
08:30:28 | 200 | 306.969µs | 192.168.1.21 | GET | /api/search/info | -
08:30:28 | 200 | 362.813µs | 192.168.1.21 | GET | /api/search/info | -
2024/11/24 08:30:28.410881 content.go:189: [Info] error fetching all videoserror fetching Videos: no such table: tblVideos
2024/11/24 08:30:28.413063 content.go:179: [Info] Request Params:#0000000200000001 - 172.16.0.4:3000 <-> 192.168.1.21:54070 - GET http://192.168.1.94:1282/api/search/info
2024/11/24 08:30:28.413173 content.go:189: [Info] error fetching all videoserror fetching Videos: no such table: tblVideos
08:30:28 | 200 | 196.466µs | 192.168.1.21 | GET | /api/search/info | -
08:30:28 | 200 | 218.082µs | 192.168.1.21 | GET | /api/search/info | -
08:30:28 | 200 | 289.27µs | 192.168.1.21 | GET | /api/search/info | -
08:30:28 | 200 | 492.276µs | 192.168.1.21 | GET | /api/download/queued-items | -
08:30:28 | 200 | 540.546µs | 192.168.1.21 | GET | /api/download/queued-items | -
08:30:28 | 200 | 586.846µs | 192.168.1.21 | GET | /api/download/queued-items | -
08:30:28 | 101 | 109.615µs | 192.168.1.21 | GET | /ws/status | -
2024/11/24 08:30:31.559460 ingest.go:34: [Info] Request Params:&{https://www.youtube.com/playlist?list=PLa4DEu_EKtlZFsVaMQ1zEpjitdz2I4ZyF false false}
2024/11/24 08:30:40.630964 network.go:314: [Info] Error Reading:EOF
error: no such table: tblChannels
error: no such table: tblPlaylists
error: no such table: tblDomains
error: no such table: tblFormats
error: no such table: tblVideos
error: no such table: tblPlaylistVideoFiles
error: no such table: tblCategories
error: no such table: tblVideoFileCategories
error: no such table: tblChannels
error: no such table: tblPlaylists
error: no such table: tblDomains
error: no such table: tblFormats
error: no such table: tblVideos
error: no such table: tblPlaylistVideoFiles
error: no such table: tblCategories
error: no such table: tblVideoFileCategories
2024/11/24 08:30:48.286550 network.go:314: [Info] Error Reading:EOF
2024/11/24 08:30:48.515252 network.go:314: [Info] Error Reading:EOF
error: no such table: tblFiles
08:30:31 | 200 | 16.956692009s | 192.168.1.21 | POST | /api/download/metadata | -
08:30:31 | 200 | 16.956714838s | 192.168.1.21 | POST | /api/download/metadata | -
08:30:31 | 200 | 16.956743835s | 192.168.1.21 | POST | /api/download/metadata | -
08:30:48 | 200 | 117.544µs | 192.168.1.21 | GET | /api/download/queued-items | -
08:30:48 | 200 | 141.982µs | 192.168.1.21 | GET | /api/download/queued-items | -
08:30:48 | 200 | 181.194µs | 192.168.1.21 | GET | /api/download/queued-items | -
2024/11/24 08:30:48.529892 content.go:179: [Info] Request Params:#0000000200000005 - 172.16.0.4:3000 <-> 192.168.1.21:54070 - GET http://192.168.1.94:1282/api/search/info
2024/11/24 08:30:48.530016 content.go:189: [Info] error fetching all videoserror fetching Videos: no such table: tblVideos
08:30:48 | 200 | 257.239µs | 192.168.1.21 | GET | /api/search/info | -
08:30:48 | 200 | 278.776µs | 192.168.1.21 | GET | /api/search/info | -
08:30:48 | 200 | 318.265µs | 192.168.1.21 | GET | /api/search/info | -
08:30:48 | 101 | 146.302µs | 192.168.1.21 | GET | /ws/status | -
2024/11/24 08:30:54.110615 network.go:346: [Info] MESSAGE VALUE: WARNING: --paths is ignored since an absolute path is given in output template
2024/11/24 08:30:54.210314 network.go:346: [Info] MESSAGE VALUE: [download] 0.0% of 255.61MiB at Unknown B/s ETA Unknown
2024/11/24 08:30:55.169269 network.go:346: [Info] MESSAGE VALUE: [download] 15.1% of 255.61MiB at 46.45MiB/s ETA 00:04
2024/11/24 08:30:56.147300 network.go:346: [Info] MESSAGE VALUE: [download] 26.7% of 255.61MiB at Unknown B/s ETA Unknown
2024/11/24 08:30:57.148298 network.go:346: [Info] MESSAGE VALUE: [download] 43.6% of 255.61MiB at 55.49MiB/s ETA 00:02
2024/11/24 08:30:58.217391 network.go:346: [Info] MESSAGE VALUE: [download] 59.0% of 255.61MiB at 34.78MiB/s ETA 00:03
2024/11/24 08:30:59.189238 network.go:346: [Info] MESSAGE VALUE: [download] 72.1% of 255.61MiB at 34.72MiB/s ETA 00:02
2024/11/24 08:31:00.144721 network.go:346: [Info] MESSAGE VALUE: [download] 85.9% of 255.61MiB at 31.77MiB/s ETA 00:01
2024/11/24 08:31:01.254671 network.go:346: [Info] MESSAGE VALUE: [download] 100% of 255.61MiB in 00:00:07 at 35.94MiB/s
2024/11/24 08:31:01.269117 network.go:346: [Info] MESSAGE VALUE: [download] 0.0% of 3.69MiB at Unknown B/s ETA Unknown
2024/11/24 08:31:01.309156 network.go:346: [Info] MESSAGE VALUE: [download] 100% of 3.69MiB in 00:00:00 at 68.65MiB/s
2024/11/24 08:31:02.025006 network.go:346: [Info] MESSAGE VALUE: [download] 100% of 3.69MiB in 00:00:00 at 68.65MiB/s
2024/11/24 08:31:02.025033 network.go:351: [Info] Error Reading:EOF
2024/11/24 08:31:02.025047 service.go:88: [Info] VideoById -1: no such table: tblVideos
pkg dowonload network - ExtractMediaContent open ../content///Videos: no such file or directory
2024/11/24 08:31:03.296271 content.go:179: [Info] Request Params:#0000000200000006 - 172.16.0.4:3000 <-> 192.168.1.21:54070 - GET http://192.168.1.94:1282/api/search/info
2024/11/24 08:31:03.296390 content.go:189: [Info] error fetching all videoserror fetching Videos: no such table: tblVideos
08:31:03 | 200 | 157.737µs | 192.168.1.21 | GET | /api/search/info | -
08:31:03 | 200 | 177.015µs | 192.168.1.21 | GET | /api/search/info | -
08:31:03 | 200 | 219.729µs | 192.168.1.21 | GET | /api/search/info | -
08:31:04 | 200 | 118.123µs | 192.168.1.21 | GET | /api/download/queued-items | -
08:31:04 | 200 | 149.196µs | 192.168.1.21 | GET | /api/download/queued-items | -
08:31:04 | 200 | 187.844µs | 192.168.1.21 | GET | /api/download/queued-items | -
08:31:04 | 200 | 120.335µs | 192.168.1.21 | GET | /api/download/queued-items | -
08:31:04 | 200 | 146.93µs | 192.168.1.21 | GET | /api/download/queued-items | -
08:31:04 | 200 | 196.462µs | 192.168.1.21 | GET | /api/download/queued-items | -
2024/11/24 08:31:08.642011 network.go:346: [Info] MESSAGE VALUE: WARNING: --paths is ignored since an absolute path is given in output template
2024/11/24 08:31:08.743315 network.go:346: [Info] MESSAGE VALUE: [download] 0.0% of 178.88MiB at 778.31KiB/s ETA 03:57
2024/11/24 08:31:09.675076 network.go:346: [Info] MESSAGE VALUE: [download] 21.7% of 178.88MiB at 46.18MiB/s ETA 00:03
2024/11/24 08:31:10.704548 network.go:346: [Info] MESSAGE VALUE: [download] 43.8% of 178.88MiB at 19.58MiB/s ETA 00:05
2024/11/24 08:31:11.679458 network.go:346: [Info] MESSAGE VALUE: [download] 64.9% of 178.88MiB at Unknown B/s ETA Unknown
2024/11/24 08:31:12.731568 network.go:346: [Info] MESSAGE VALUE: [download] 91.0% of 178.88MiB at 52.58MiB/s ETA 00:00
2024/11/24 08:31:13.540154 network.go:346: [Info] MESSAGE VALUE: [download] 100% of 178.88MiB in 00:00:04 at 36.73MiB/s
2024/11/24 08:31:13.552857 network.go:346: [Info] MESSAGE VALUE: [download] 0.0% of 3.40MiB at Unknown B/s ETA Unknown
2024/11/24 08:31:13.610898 network.go:346: [Info] MESSAGE VALUE: [download] 100% of 3.40MiB in 00:00:00 at 48.54MiB/s
2024/11/24 08:31:14.191175 network.go:346: [Info] MESSAGE VALUE: [download] 100% of 3.40MiB in 00:00:00 at 48.54MiB/s
2024/11/24 08:31:14.191196 network.go:351: [Info] Error Reading:EOF
2024/11/24 08:31:14.191209 service.go:88: [Info] VideoById -1: no such table: tblVideos
pkg dowonload network - ExtractMediaContent open ../content///Videos: no such file or directory
2024/11/24 08:31:15.305456 content.go:179: [Info] Request Params:#0000000100000004 - 172.16.0.4:3000 <-> 192.168.1.21:54069 - GET http://192.168.1.94:1282/api/search/info
2024/11/24 08:31:15.305572 content.go:189: [Info] error fetching all videoserror fetching Videos: no such table: tblVideos
08:31:15 | 200 | 145.66µs | 192.168.1.21 | GET | /api/search/info | -
08:31:15 | 200 | 169.17µs | 192.168.1.21 | GET | /api/search/info | -
08:31:15 | 200 | 212.615µs | 192.168.1.21 | GET | /api/search/info | -
BTW, It's Jan ;-)
Hey Jan, :)
thanks for the update! It seems like one error is fixed and thumbs are going into the right location. Could you please confirm if the videos are also getting stored in the parent folder for the thumbnails?
I really appreciate your time and efforts on this! :-) Thank you!
Update: If the video files are also being stored in the content directory, kindly try the following -
for the error that we have now, it looks like the database is not in place for the application, although it is shipped initialized within the docker image itself.
This could be because the bind-mount did not bind correctly onto the physical location. Can you try the below steps, (this would really help identify the problem) -
Once the database is in place, do try to download through the streamsphere now and it should work!
No, as of now, no video is to be found anywhere, only the thumbnails are stored into
/content/youtube.com/[name of playlist]/Videos/Thumbnails/*.webp
/Videos
folder contains nothing but the /Thumbnails
subfolder.
The database is located in
/srv/dev-disk-by-uuid-5b67514d-485e-4306-873e-b1cbb54ccf99/Config/streamsphere/db-data/streamsphere.db
but it is 0 Bytes in size, so nothing seems to be saved to it after job is done. It's been created by the app as soon as it started.
Should I do the steps anyway?
No no, pls dont perform the steps since the video files are not in place. The data base is being created by the application which is incorrect, thats why it is 0 bytes.
Can you pls try to remove the streamsphere & streamsphere-content containers, their images, and the docker bind-mounts as well and then try again with the "docker compose up -d".
The path for the videos is in the golang binary file, the videos should be downloaded into the /content/youtube.com/[name of playlist]/Videos/ directory.
Also just to confirm you are using an AMD64 machine or ARM64?
Ok, I have erased everything, including images, containersm, etc and redeployed from scratch. Now all I see is:
http://192.168.1.94:1282
http://192.168.1.94:1282
http://192.168.1.94:1288
Connected to database.
┌───────────────────────────────────────────────────┐
│ Streamsphere │
│ Fiber v2.52.5 │
│ http://127.0.0.1:3000 │
│ (bound on host 0.0.0.0 and port 3000) │
│ │
│ Handlers ............ 35 Processes ........... 1 │
│ Prefork ....... Disabled PID ................ 11 │
└───────────────────────────────────────────────────┘
and the app doesn't move from there, nor I can open any UI. Tried to restart, but it just repeated the same behavior.
A 0-bytes .db file has been created, as well as a /content
folder, but I cannot use it in any way.
Yes, I use AMD64 and I do everything from Portainer, my machine runs Open Media Vault 6.
Hi Jan,
This is quite unexpected! let me get back to you in sometime, I will make an omv6 + portainer installation and try to replicate the issue you are facing. Sorry for the troubles! and I really appreciate your time and effort on this! :)
Hey,
If you can make just one last try, and I understand its asking a bit too much but can you pls remove container, bind mount, images and once try with the following docker-compose:
services:
streamsphere:
image: streamsphere/streamsphere:latest
# image: streamsphere/streamsphere:latest-arm
container_name: streamsphere
restart: unless-stopped
ports:
- 1282:3000
environment:
HOST_IP: "192.168.1.94"
APPLICATION_PORT: "1282"
CONTENT_PORT: "1288"
volumes:
- db-data:/app/database/db
- content-data:/app/content
content:
image: streamsphere/streamsphere-content:latest
# image: streamsphere/streamsphere-content:latest-arm
container_name: streamsphere-content
restart: unless-stopped
ports:
- 1288:3500
volumes:
- content-data:/content
volumes:
db-data:
content-data:
networks:
default:
external:
name: omv1
Thank you! If you're not able to do this test I will totally understand!
I've just deployed Streamsphere, I've added the first random playlist YT proposed me and the UI actually showed the downloads were happening. But after finishing, both Playlists and Videos sections were empty. Also the content folder was empty. The only thing that actually happened was the video thumbnails stored in the config folder. This is my compose, what's wrong?
This is the streamsphere-content log:
And this is the last bit of the main app log: