Closed gityopr closed 2 years ago
Can you give me the output of ls -l
? it updates based on the modification timestamps of the file, so that needs to be newer than dir2cast
Thanks for the quick reply!
pi@rpi:~/podcast-server/kdbookecast $ ls -l
total 2164536
-rw-rw-r-- 1 pi pi 1203 Dec 21 18:34 502.html
-rwxr--r-- 1 pi pi 3767269 Dec 21 18:44 'A Full Life by Paulo Bacigalupi.m4a'
-rw-r--r-- 1 pi pi 295773 Dec 21 18:39 'Carl Sagan - The Demon-Haunted World (Cary Elwes, Seth MacFarlane).jpg'
-rwxrw-r-- 1 pi pi 498811479 Dec 21 18:34 'Carl Sagan - The Demon-Haunted World (Cary Elwes, Seth MacFarlane).mp3'
-rwxrw-r-- 1 pi pi 544397046 Dec 21 18:35 colorless-tsukuru-tazaki-and.mp3
-rw-rw-r-- 1 pi pi 96 Dec 21 18:34 composer.json
-rwxrwxr-x 1 pi pi 8908 Dec 21 18:35 dir2cast.ini
-rw-rw-r-- 1 pi pi 66709 Dec 21 18:34 dir2cast.php
drwxrwxr-x 2 pi pi 4096 Dec 21 18:34 getID3
-rwx-w---- 1 pi pi 406827669 Dec 21 18:35 'How Beautiful We Were.mp3'
-rwxrw-r-- 1 pi pi 22074 Dec 21 18:35 image.jpg
-rw-rw-r-- 1 pi pi 780 Dec 21 18:35 index.html
-rw-r--r-- 1 pi pi 31116 Dec 21 18:39 'Klara and the Sun.jpg'
-rwxrw-r-- 1 pi pi 587253078 Dec 21 18:36 'Klara and the Sun.m4a'
-rw-r--r-- 1 pi pi 115349 Dec 21 18:39 'Love Is an Ex-Country.jpg'
-rwxrw-r-- 1 pi pi 174817435 Dec 21 18:36 'Love Is an Ex-Country.m4a'
drwxr-xr-x 2 pi pi 4096 Dec 21 18:39 temp
I touch'd "A Full Life by Paulo Bacigalupi.m4a" since that was the one I was testing. That caused the script to pick it up and it appeared in my client apps shortly after. However, pasting a new file which has a newer modification date into the directory didn't update it.
pi@rpi:~/podcast-server/kdbookecast $ ls -la
total 2171220
drwxrwxr-x 4 pi pi 4096 Dec 22 08:19 .
drwxr-xr-x 14 pi pi 4096 Dec 20 22:20 ..
-rw-rw-r-- 1 pi pi 1203 Dec 21 18:34 502.html
-rwxr--r-- 1 pi pi 3767269 Dec 22 08:13 'A Full Life by Paulo Bacigalupi.m4a'
-rw-r--r-- 1 pi pi 6824161 Dec 22 08:19 'Alien (Preview).mp3'
-rw-r--r-- 1 pi pi 295773 Dec 21 18:39 'Carl Sagan - The Demon-Haunted World (Cary Elwes, Seth MacFarlane).jpg'
-rwxrw-r-- 1 pi pi 498811479 Dec 21 18:34 'Carl Sagan - The Demon-Haunted World (Cary Elwes, Seth MacFarlane).mp3'
-rwxrw-r-- 1 pi pi 544397046 Dec 21 18:35 colorless-tsukuru-tazaki-and.mp3
-rw-rw-r-- 1 pi pi 96 Dec 21 18:34 composer.json
-rwxrwxr-x 1 pi pi 8908 Dec 21 18:35 dir2cast.ini
-rw-rw-r-- 1 pi pi 66709 Dec 21 18:34 dir2cast.php
-rwxrw-r-- 1 pi pi 6148 Dec 21 18:39 .DS_Store
drwxrwxr-x 2 pi pi 4096 Dec 21 18:34 getID3
-rwx-w---- 1 pi pi 406827669 Dec 21 18:35 'How Beautiful We Were.mp3'
-rwxrw-r-- 1 pi pi 22074 Dec 21 18:35 image.jpg
-rw-rw-r-- 1 pi pi 780 Dec 21 18:35 index.html
-rw-r--r-- 1 pi pi 31116 Dec 21 18:39 'Klara and the Sun.jpg'
-rwxrw-r-- 1 pi pi 587253078 Dec 21 18:36 'Klara and the Sun.m4a'
-rw-r--r-- 1 pi pi 115349 Dec 21 18:39 'Love Is an Ex-Country.jpg'
-rwxrw-r-- 1 pi pi 174817435 Dec 21 18:36 'Love Is an Ex-Country.m4a'
drwxr-xr-x 2 pi pi 4096 Dec 22 08:14 temp
This is the situation with the folder now, the server lists "A Full Life....m4a" but not "Alien (Preview).mp3". Here's the output of ls -l inside the temp folder:
pi@rpi:~/podcast-server/kdbookecast $ ls -l temp/
total 40
-rw-r--r-- 1 pi pi 268 Dec 21 18:39 '1205de5de8d7ad1fc2ac4079e146bfc1__Carl Sagan - The Demon-Haunted World (Cary Elwes, Seth MacFarlane).mp3__data'
-rw-r--r-- 1 pi pi 261 Dec 21 18:39 '29fb0b475b5c5980f6afcd5e08ce3f22__Klara and the Sun.m4a__data'
-rw-r--r-- 1 pi pi 104 Dec 22 08:14 '513726ad29a49b5aa9324880afb8c615__A Full Life by Paulo Bacigalupi.m4a__data'
-rw-r--r-- 1 pi pi 109 Dec 21 18:39 '93ec60a54c683cd5d310584ff36cc430__How Beautiful We Were.mp3__data'
-rw-r--r-- 1 pi pi 1241 Dec 21 18:39 '95822048accbb8e847e963dd005f5508__Love Is an Ex-Country.m4a__data'
-rw-r--r-- 1 pi pi 77 Dec 22 08:14 'ae5150c0a97439f7a4327499bf2aea36__._A Full Life by Paulo Bacigalupi.m4a__data'
-rw-r--r-- 1 pi pi 9793 Dec 22 08:27 bf49df1dd863708a9101bb03671c5556__config_www.xml
-rw-r--r-- 1 pi pi 313 Dec 21 18:39 cfb467a3fad5b73a1e9808c74ac9b5ed__colorless-tsukuru-tazaki-and.mp3__data
Thanks for your help!
Edit: Thought the code button would make it prettier, fixed that!
Try touching all of them in sequence, as they're mostly older than dir2cast. I wonder if copying them is preserving the date from the source folder? Then wait. Few mins before adding more
I think i see a separate bug, it shouldn't be including the .
file for A Full Life
the config_www.xml temp file (i assume that's the feed itself) looks like it contains everything, is it just being browser cached? do you need to do a hard refresh in the browser?
dir2cast is designed to wait a bit after you update so that it isn't adding broken MP3s to the feed (as you copy them in), but waiting 5 mins solves that - or you can fiddle with that in the config
basically if that .xml temp file updates (and contains all the files) then the problem is probably caching in your browser.
the following two settings apply to your use case
; *** THESE SHOULD BE LEFT AS THEY ARE - unless you have a good reason. ***
; Number of seconds for which the cache file is guaranteed valid. You can
; only specify this in the dir2cast.ini that is in the same folder as
; dir2cast.php. Defaults to 5
;MIN_CACHE_TIME = 5
; Time-to-live (Expiry time) of the feed
; This defaults to 60 minutes
;TTL = 60
a new feed is valid of a minimum of 5 seconds before it will even try to regenerate. Also, it declares the feed as being current for an hour so feed readers may not refresh it for an hour unless you force them to. This includes HTTP caches and browsers, potentially also nginx depending how it's configured. (tacking ?saidsa junk on the end usually bypasses this if you want to test.)
Anyway, you can reduce these if you want.
there is also a URL ?force=password where you can make it refresh anyway, that's if the cache files are definitely not right.
; The password to use on the 'force=<password>' part of the URL in order to
; force the RSS feed to be regenerated (bypassing and replacing the cached
; copy). If this is empty then this feature is disabled.
;FORCE_PASSWORD =
Thanks, touching all the media files and confirming the .xml in that folder did the trick. I knew it would be something silly! Thank you.
Still technically a bug though (I didn't intend for it not to work properly if you copied dir2cast in AFTER your media files),, so I will try to improve the experience here!
Yeah something is definitely up with the stock swag docker container and dir2cast. Tried spinning up a new webserver on a Debian VM instead of a Raspberry Pi. I also waited about an hour to add files to the directory.
The server works great but dir2cast needs that reset link or clearing the temp folder to update. Does it rely on some PHP extensions or anything like that?
Yeah something is definitely up with the stock swag docker container and dir2cast. Tried spinning up a new webserver on a Debian VM instead of a Raspberry Pi. I also waited about an hour to add files to the directory.
The server works great but dir2cast needs that reset link or clearing the temp folder to update. Does it rely on some PHP extensions or anything like that?
Only the XML extension which you must have if it worked at all. It does rely on the file modification times so if your file system is FAT32 or something instead of ext3 or ext4 it might not work. I’ve never tried it on fat32
I have been working on some more tests to see if I can reproduce and fix the problem. Thanks for reporting it.
i have done some work on the caching which may solve your problem, would love to know if they do
would also be useful if you could run the test suite for me on your Pi
Sorry for the late reply, I've been away from this project because of the holidays. How do I test the test suite?
To run the test suite,
apt install php
pecl install xdebug
composer
is installed (not sure if you can get that through apt
but if you can't, these instructions seem reasonablegit clone
dir2cast onto your Pi from GitHubcd test
), run composer install
./run.sh
Hi @gityopr hoping to hear back from you, or perhaps give instructions so I can have a go :) but I have a hunch it's a twist specific to your particular combination and I might not be able to replicate it
I'm sorry for the delay on this! I believe part of the issue is that I'm running my webserver out of a docker container. SWAG by Linuxserver. Last time I looked at this I had a lot of trouble figuring out how to get composer running in that container. I'll give it a bit more work this weekend.
Thanks for not closing this issue!
Last time I looked at this I had a lot of trouble figuring out how to get composer running in that container.
I would have thought you'd do something like this:
swag
and docker ps
shows it's running)docker exec -it swag /bin/bash
- this will open a shell into the containercd /config
- from the example docs i've seen, this seems to be the path inside the container where the files are mounted incd ...
- find where the dir2cast checkout isI have the same problem. Downloading a video with youtube-dl-server use the date of video. Also touch the file doesn't help. Even deleting a video still shows the video in my Podcast app. Solution is deleting dir2casts temp folder. Fow now I will run a 3rd container with a cronjob deleting the temp folder, if file count has been changed.
@helmut72 can you try setting MIN_CACHE_TIME=0 and MIN_FILE_AGE=0 in dir2cast.ini, it should effectively disable the cache altogether
I would still like to know more about the filesystem and the file dates that you have which are causing this behaviour - ideally a test case that I can work with so that I can reproduce the problem?
Hi, I use a standard Ubuntu 22.04 with Docker installed from docker.com, Apache with PHP. Nothng special with one exception: to reduce wear on a Raspberry I have set noatime option in fstab for ages, so I do it also on my server with SSD. I have moved my collection from one folder to the video folder of dir2cast. File dates are different. From 2019 to may 2022. Then I have tested if dir2cast recognize them. Can test your suggested settings after work.
Thank you, I appreciate the feedback. I have an idea for a fix which will enable this to work with your use cases.
I did not realise how popular it would be to copy "old" files into a podcast folder (I don't do this personally), but I will add some code to refresh the feed if the number of files changes.
can you try setting MIN_CACHE_TIME=0 and MIN_FILE_AGE=0 in dir2cast.ini
This fix one problem, using touch file.mp4
add the file in my Podcast app. But deleting a file it is still shown in my Podcast app. Also files with old dates won't be added.
I did not realise how popular it would be to copy "old" files into a podcast folder
When someone uploaded a video on Youtube in 2019, file date becomes 2019 on my harddrive. I use youtube-dl-server to get my content easily.
Also try wget:
wget https://jsoncompare.org/LearningContainer/SampleFiles/Video/MP4/sample-mp4-file.mp4
File date is 2021 on the harddrive. dir2cast never find it.
Thanks. I understand your situation a lot better now. I really appreciate you taking the time to add details to the bug report.
I will come up with a solution for this.
@gityopr or @helmut72 if you'd care to try the #58 to see if it fixes your problem? if so, i'll merge this and add it to the official release
Looks good to me. Adding, deleting, adding again, deleting again, using touch
before adding a 3rd time... all behaves like expected. Great, thanks!
v1.33 has been released which incorporates these changes. Thanks for your help!
Hi, I can't get dir2cast to update the feed when a new file is added to its directory. Deleting a file from the directory also doesn't cause any changes.
If I delete the contents of /temp then it refreshes appropriately but I'd prefer not to have to do this every time, if possible.
Details: I'm running dir2cast on a Raspberry Pi 4 in the /www folder of a SWAG docker container. Otherwise my .ini file is stock.