K-vanc / Tempest-EPG-Generator

The fastest XMLTV formatted EPG Generator with GUI and the lowest memory usage
https://hub.docker.com/r/kvanc/tempest_epg
Other
100 stars 17 forks source link

mmap() failed: Cannot allocate memory #62

Closed stuckinthe closed 7 months ago

stuckinthe commented 7 months ago

I am having an issue with memory on two of my systems. Both are headless. One gives me the above error (256MB) on a VPS and on my Rpi with 1GB it stops in the middle of a xumo grab of all channels 300+ for 2 days. I am able to run it on another computer that has 8GB of memory. I just don't always keep it running. On all systems I have turned the max ram for php to -1 or off. Does Tempest consume a fair amount of RAM during a grab?

Also, I am missing some channels in xumo and doing the key update doesn't fix them.

K-vanc commented 7 months ago

hi @stuckinthe

first of all, i have no idea what is mmap and can confirm it is not a required/direct used module by Tempest. xumo is a all channels data is provided for each request website, this means if there are 100 channels, for grabbing 1 channel, you are receiving data of other 99 also. This means higher memory usage will be required for sure. During my testings, I didnt notice an issue for more than default mem(128M for php.ini) is needed so didnt mention but it is impossible for me test all zip codes and see if all ok on maximum days grabbing.

I made a quick search on google for mmap and it seems some swap(low or no swap) issue may trigger this error.

On all systems I have turned the max ram for php to -1 or off

I dont know what is above command but i always set memory from php.ini instead of system since Tempest only using php memory. You can find below link for some examples;

https://github.com/K-vanc/Tempest-EPG-Generator/issues/54#issuecomment-1741850211

Does Tempest consume a fair amount of RAM during a grab?

you can see my grabbing which is fine but as I said, it depends on what you receive as response. I cannot test all zip codes 1 by 1. If there is a higher memory needed data, users has to adjust their server.

image

Also, I am missing some channels in xumo and doing the key update doesn't fix them.

Again, i cannot check all zip codes 1 by 1 to see all channels are grabbed/generated. Instead of "some", if you write in which zip code, you are missing which channel, I can test if channel is there or I need to revise siteconfig.

stuckinthe commented 7 months ago

I did a google search and I saw that the error was related to swap. My other config file has no issue so I wanted to see if it could have been another reason besides the swap. I'm running in an openvz container so I will need to enable vswap. I rebooted my PI and it isn't timing out. I enabled the error reporting log error_reporting and if still has an issue I will post it here.

I was using zip code 10000 for xumo and it was missing the CBS channels and Mixable I believe. No worries. There are a lot of SiteConfig to keep track of.

Do you know how to get the SamsungTVPlus guide data? I thought it said on here it was in the Multi Nations folder in the distro file and you need too use the SamsungTVPlus key but this didn't work for me. I looked in the info files and I didn't see any directions.

K-vanc commented 7 months ago

Hi, I Will have a look for the channel when i have time. (But publishing may take some time because i am working on an emergency update)

For Samsung, i dont have direct source since it is not available in my region and never recognized to check it while i am on my PC with vpn but it should be available in i.mjh.nz siteconfig in Multi Nation folder if still active;

https://github.com/K-vanc/Tempest-EPG-Generator/blob/main/Siteconfigs/Multi%20Nation/i_mjh_nz_codes.txt

stuckinthe commented 7 months ago

It's missing the UFC channel which I am mostly looking to get.

K-vanc commented 7 months ago

I dont see any epg page on samsung plus website. It is located behind some login? if you know the source url or location, you can send me so I will have a look when I am done with my update

stuckinthe commented 7 months ago

The Xumo epg channels (CBS, UFC, etc) are working. I needed to do a channel update.

Samsung TV plus isn't behind any login. Just go to their website and you will see it. I generated the channels per above. All is good.

stuckinthe commented 7 months ago

I spoke too soon. The i_mjh_nz site gave me a 404 on Kitchen Nightmares.

stuckinthe commented 7 months ago

https://www.samsungtvplus.com/en

The epg is below the show.

K-vanc commented 7 months ago

So this means there is no problem with xumo anymore?

I.mjh.nz is a xml sharing website(probably some kind of ftp server). Tempest siteconfig grabs from already generated xml files. If there is no channel in the xml file, you Will get nothing

So for Samsung, for sure i Will need vpn because i see nothing when i open website

stuckinthe commented 7 months ago

xumo is good.

I believe the link to the xml file is bad as I get a 404 so I assume it can't find the file to grab. I tried other channels and they gave me a 404.

K-vanc commented 7 months ago

xumo is good.

I believe the link to the xml file is bad as I get a 404 so I assume it can't find the file to grab. I tried other channels and they gave me a 404.

You can check from here. Tempest siteconfig only simplifies it.

https://i.mjh.nz/

stuckinthe commented 7 months ago

It just downloads the whole file and adds it to the whateveryoucall.xml file in the config.xml file you create.

K-vanc commented 7 months ago

It just downloads the whole file and adds it to the whateveryoucall.xml file in the config.xml file you create.

With a small correction. Tempest is not downloading whole file but reads it from the stream, then parsing each element same as grabbing from any html source and re-assigning to same place in standard xml structure. Thats why you are able to receive only selected channels

stuckinthe commented 7 months ago

I think the siteconfig has the wrong url to the stream as I would assume the 404 is because it can't find the file.

K-vanc commented 7 months ago

Maybe you should rechannel create after creating fresh userkey list??

https://github.com/K-vanc/Tempest-EPG-Generator/blob/main/Siteconfigs/Multi%20Nation/Info%20for%20tvguide_com%20-%20i_mjh_nz%20-%20epgshare01.md

stuckinthe commented 7 months ago

It gives you the error but it actually pulls what is available. Probably because I asked for too many days. I will close as no issue.

stuckinthe commented 7 months ago

It must've pulled the epg data from another source as it said no shows grabbed.

K-vanc commented 7 months ago

hi @stuckinthe

If you check the website, you will see that there are several xml, html, text, m3u etc files. Siteconfig is only looking for 1 of them. If you write in which userkey/channel you are having issue, I can look. I randomly tested 2 channels and both grabbed without error.

Maybe something changed in the filenames so it is giving 404 error

stuckinthe commented 7 months ago

I used the key of SamsungTVPlus instead of SamsungTVPlus/ the / is important for anyone else who reads this.

Closed for good this time.