Closed DadRe-dmt closed 5 months ago
Hello, thanks for that info. Don't worry about bricking it, it's safe to update, it has two partitions app0/app1 for firmware, during the updating, it only write the other partition, if update is not success, it will boot from the origin partition.
I've found the issue, related to Stock app, and will offer another update as soon as possible.
@DadRe-dmt Please try the updates, thank you.
Problem is still the same. After a few seconds the device is rebooting. I tried all new firmware up till 37. The last version that doesn't trigger the reboot, is version 31.
Problem seems to occur on every single FW update since 31.
I also notice that after updating any FW higher than 31, the time (clock) stays at 0:00 , it doesn't seem to sync.
Are you connected to the internet? Because I face the same problem if the device has Wifi but no internet access.
Problem is still the same. After a few seconds the device is rebooting. I tried all new firmware up till 37. The last version that doesn't trigger the reboot, is version 31.
Problem seems to occur on every single FW update since 31.
I also notice that after updating any FW higher than 31, the time (clock) stays at 0:00 , it doesn't seem to sync.
OK, I'll check it again, is it Bitcoin app you used when the reboot occurs? Let's focus on the latest V3.2.37, the time should be updated. Also I have tried bitcoin/stock/weather, but no reboot was observed within a few hours. If that's possible, can you share the settings of the App?
Are you connected to the internet? Because I face the same problem if the device has Wifi but no internet access.
Hi, which App has the same problem? I'll try reproduce it.
I have tried with my phone's hotspot, close the internet, but haven't found it will reboot, in weather/stock/bitcoin apps.
Problem is still the same. After a few seconds the device is rebooting. I tried all new firmware up till 37. The last version that doesn't trigger the reboot, is version 31. Problem seems to occur on every single FW update since 31. I also notice that after updating any FW higher than 31, the time (clock) stays at 0:00 , it doesn't seem to sync.
OK, I'll check it again, is it Bitcoin app you used when the reboot occurs? Let's focus on the latest V3.2.37, the time should be updated. Also I have tried bitcoin/stock/weather, but no reboot was observed within a few hours. If that's possible, can you share the settings of the App?
It is not the Bitcoin app in specific. When setting 'autostart' to clock, the reboot is happening as well. I can make some screenshots, which part of the interface ?
Are you connected to the internet? Because I face the same problem if the device has Wifi but no internet access.
Hi, which App has the same problem? I'll try reproduce it.
I have tried with my phone's hotspot, close the internet, but haven't found it will reboot, in weather/stock/bitcoin apps.
No app, you just have to boot up and wait. After maximum 2 minutes it crashes and reboots itself this will loop forever. However my default startup app is pictures. I assume that it has to do with having wifi connected but not having internet access because of firewall blocking it. I have two devices maybe I will fiddle around with the other one.
Problem is still the same. After a few seconds the device is rebooting. I tried all new firmware up till 37. The last version that doesn't trigger the reboot, is version 31. Problem seems to occur on every single FW update since 31. I also notice that after updating any FW higher than 31, the time (clock) stays at 0:00 , it doesn't seem to sync.
OK, I'll check it again, is it Bitcoin app you used when the reboot occurs? Let's focus on the latest V3.2.37, the time should be updated. Also I have tried bitcoin/stock/weather, but no reboot was observed within a few hours. If that's possible, can you share the settings of the App?
It is not the Bitcoin app in specific. When setting 'autostart' to clock, the reboot is happening as well. I can make some screenshots, which part of the interface ?
I've tried to set the autostart to "Clock", but seems work all good for a half hour now.
How often does the reboot occurs? What is the version before the update to 37? Did you tried with a factory reset?
Are you connected to the internet? Because I face the same problem if the device has Wifi but no internet access.
Hi, which App has the same problem? I'll try reproduce it. I have tried with my phone's hotspot, close the internet, but haven't found it will reboot, in weather/stock/bitcoin apps.
No app, you just have to boot up and wait. After maximum 2 minutes it crashes and reboots itself this will loop forever. However my default startup app is pictures. I assume that it has to do with having wifi connected but not having internet access because of firewall blocking it. I have two devices maybe I will fiddle around with the other one.
OK, does the firewall blocks all tcp/udp connections? Can try close the firewall to check if it the reboot still occurs?
I tested to set the autostart to "Pictures", but seems works too, with no reboot in 10 minutes. Is it 3.2.37 version?
Are you connected to the internet? Because I face the same problem if the device has Wifi but no internet access.
Interesting! The devices have (strictly filtered) internet connection, BUT........... the problem indeed occurs when not allowing specific outgoing connections. When the device boots, it keeps polling 213.188.196.246 (port 80) which seems part of worldtimeapi.org platform. This was also the case with firmware 31 and before, but besides polling to worldtimeapi.org it also used regular NTP 123 for time syncing. Blocking the http connection to worldtimeapi.org did not make the device boot and the time was syncing without a problem.
In my logging i notice that firmwares after 31 don't seem to use NTP protocol for time syncing anymore. It looks like connecting to worldtimeapi.org during boot became mandatory to sync time ? If not allowed during boot, the device will enter a bootloop.
When i block outgoing http connections to 213.188.196.246 AFTER the reboot happened, the device remains up and running. Until i reboot the device again, the problem start happening again.
I rather use trusted NTP servers for unknown devices, than outgoing HTTP connections to some timesync platform. This is why i blocked the outgoing HTTP connections to worldtimeapi.org
@GeekMagicClock Hope this helps to resolve the issue.
"It is not recommended that this API be used for commercial applications. The API can go down from time-to-time, for relatively long periods. It is provided with no SLA, no guarantees, and no direct funding.
While the data we return should be accurate, no guarantees can be made. We don't manage the infrastructure or network, so cannot be 100% certain about accuracy. We update the time details at the last possible moment in our application before returning the data to you, and we find that responses are generated in just a few ms, but anything sub-second should realistically be ignored.
Please note that there is a fair-use policy in place.
Therefore, this API should not be used:
in "real-time" situations where absolutely accurate timing data is required as an alternative to ntp as an alternative to calculating local time changes (eg. making a request every second) "
If the Worldtime api indeed goes down for longer periods of time, or having other longlasting issues... It might possibly cause al the GeekMagicClock devices (worldwide?) to go into bootloop after booting ? (Single point of failure)
Interesting findings. I will not allow IoT devices to use internet access if they do not need to. NTP is SNATed for not bypassing my DNS sinkhole filter but working.
Interesting findings. I will not allow IoT devices to use internet access if they do not need to. NTP is SNATed for not bypassing my DNS sinkhole filter but working.
Interesting findings. I will not allow IoT devices to use internet access if they do not need to. NTP is SNATed for not bypassing my DNS sinkhole filter but working.
Same here. Very restrictive with especially IoT devices. The majority allows all........ i guess we are not the majority. I'm blocking any outgoing connection unless required and 'trusted'.
Since 32 fw , something seems changed. I run local NTP and even with trusted external NTP's , Since 32 FW, whatever NTP server i fill up ......... the device just completely stopped polling 123 NTP during boot. Only worldtimeapi 80 is getting polled over and over again, and seems mandatory for proper boot. After a few denies, the device is booting again. Seems like the NTP option in the interface is not operational or completely ignored. I tested it on any of the 9 devices i own, all similar result.
Quite annoying as i miss al the wonderfull updates/improvement on these great little devices. Because great little device it is! I just love it. I mainly use these devices as crypto tickers, it's by far the best project in this niche market.
Props to the dev! @GeekMagicClock
@DadRe-dmt @tinju87 Thank you all, that is quite interest finding. I think I'm one of the majority :). I did have made some time sync strategies to make sure time could be updated correctly, http or udp (NTP), async or sync. It maight be a mistake that I ignored the NTP sync strategy during my debug since 32 and later, but I can't remember the reason.
According to your findings, I think this part need to be reviewed and rewrite again. By default, udp NTP would be the first used to sync time, if that fails(caused by firewall etc by no intent), http time sync would be used as a backup method to get time correctly.
Very much appreciated for your support.
I have restore the UDP NTP time update method, and the worldtimeapi as a http backup. UDP time sync interval will be 5minutes if the first sync is success, and it will try your ntp server first (if set), but if fails or not set, it will try 5 NTP servers inside["ntp.aliyun.com", "time.windows.com", "pool.ntp.org", "europe.pool.ntp.org", "asia.pool.ntp.org"], only when the UDP method fails, the http would be used, if the first http sync fails, would try 10 seconds a time, after the first successul http sync, the interval switchs to 5 minutes. But I don't have a firewall to control all the flows, I can't fully reproduce your situation, hope this version will work with luck.
For future updates, may be more http servers would be added as a backup. Please try if you are convinent, thanks again.
I have restore the UDP NTP time update method, and the worldtimeapi as a http backup. UDP time sync interval will be 5minutes if the first sync is success, and it will try your ntp server first (if set), but if fails or not set, it will try 5 NTP servers inside["ntp.aliyun.com", "time.windows.com", "pool.ntp.org", "europe.pool.ntp.org", "asia.pool.ntp.org"], only when the UDP method fails, the http would be used, if the first http sync fails, would try 10 seconds a time, after the first successul http sync, the interval switchs to 5 minutes. But I don't have a firewall to control all the flows, I can't fully reproduce your situation, hope this version will work with luck.
For future updates, may be more http servers would be added as a backup. Please try if you are convinent, thanks again.
1 Manually set a NTP server in this case : ntp1.trans-ix.nl
2 Manual reboot/power off the device.
Result : Manually entered NTP server is not being used during boot. Bootloop starts when during the boot, the device starts polling a load of ntp servers from various timepools, except the one (server) that is defined manually.(which i allow in my firewall) My firewall is blocking any non defined connections. As soon as the device is unable to connect NTP 123 for a few attempts, the device starts rebooting.
The bootloop can be stopped when after reboot you click the save button again, at manual NTP server interface. It immediately polls the manually defined NTP server, the device succesfully syncs with the defined ntp server and stop rebooting. It just seems to forget the settings or at least not use it, when the device reboots/ or power off/on.
I'm not sure why the device starts rebooting when certain "NTP 123 / worldtimeapi" connections are blocked or simply not available. That behaviour was not happening with FW 31 and before. Is it a routine ? / mandatory condition , or is the device actually crashing on those events ?
Basically the problem is similar as before. Whenever http and/or 123 udp connections are blocked/unavailable for a few polls, the device reboots.
I will add a screenshot to show what connections happen during bootloop. and what happens after re-clicking the save button.
Green connections are when i click "save" at the devices 'time' interface. Bootloop immediately stops. (until manual reboot)
The red connections are blocked (undefined) connections during boot and result in bootloop of the device, as manually set NTP server is not being used.
I still get the reboot loop with FW_SmallTV-PRO_V3.2.38EN. Just disable the reboot. It is not necessary at all to reboot.
I still get the reboot loop with FW_SmallTV-PRO_V3.2.38EN. Just disable the reboot. It is not necessary at all to reboot.
Disabling/stopping the 'reboot' should be the no.1 priority indeed. That's the core problem.
Other issues seem minor in comparison to the bootloop.
Would be nice if the devices uses the manually entered NTP server during boot, but definately should not result in reboot when it fails doing so. Same for the fallback worldtimeapi scenario, the device shouldn't reboot when failing to connect.
1 Manually set a NTP server in this case : ntp1.trans-ix.nl
2 Manual reboot/power off the device.
Result : Manually entered NTP server is not being used during boot. Bootloop starts when during the boot, the device starts polling a load of ntp servers from various timepools, except the one (server) that is defined manually.(which i allow in my firewall) My firewall is blocking any non defined connections. As soon as the device is unable to connect NTP 123 for a few attempts, the device starts rebooting.
The bootloop can be stopped when after reboot you click the save button again, at manual NTP server interface. It immediately polls the manually defined NTP server, the device succesfully syncs with the defined ntp server and stop rebooting. It just seems to forget the settings or at least not use it, when the device reboots/ or power off/on.
I'm not sure why the device starts rebooting when certain "NTP 123 / worldtimeapi" connections are blocked or simply not available. That behaviour was not happening with FW 31 and before. Is it a routine ? / mandatory condition , or is the device actually crashing on those events ?
Basically the problem is similar as before. Whenever http and/or 123 udp connections are blocked/unavailable for a few polls, the device reboots.
I will add a screenshot to show what connections happen during bootloop. and what happens after re-clicking the save button.
1 Manually set a NTP server in this case : ntp1.trans-ix.nl
2 Manual reboot/power off the device.
Result : Manually entered NTP server is not being used during boot. Bootloop starts when during the boot, the device starts polling a load of ntp servers from various timepools, except the one (server) that is defined manually.(which i allow in my firewall) My firewall is blocking any non defined connections. As soon as the device is unable to connect NTP 123 for a few attempts, the device starts rebooting.
The bootloop can be stopped when after reboot you click the save button again, at manual NTP server interface. It immediately polls the manually defined NTP server, the device succesfully syncs with the defined ntp server and stop rebooting. It just seems to forget the settings or at least not use it, when the device reboots/ or power off/on.
I'm not sure why the device starts rebooting when certain "NTP 123 / worldtimeapi" connections are blocked or simply not available. That behaviour was not happening with FW 31 and before. Is it a routine ? / mandatory condition , or is the device actually crashing on those events ?
Basically the problem is similar as before. Whenever http and/or 123 udp connections are blocked/unavailable for a few polls, the device reboots.
I will add a screenshot to show what connections happen during bootloop. and what happens after re-clicking the save button.
Hi, yes, still need to modify the user defined NTP server, just too late to take effect in the current firmware, or next reboot will take effect.
And the reboot was caused by a bug in the http sync method, I'm working on this, should be soon to have another test version.
Thanks again.
When looking at the connection flow, it's indeed happening after several failed http polls of worldtimeapi.
During boot routine, first it tries 5 x to connect to a NTP server, then fallback to http syncing. After several failed http sync attempts (10 - 20x) ,the bug occurs and the device reboots and repeats the procedure. Until it's able to either sync with a NTP server or by http worldtimeapi, then the bootloop stops immediately.
I stated this was not happening in 31 and before, but the problem is also present in earlier FW versions. I just didn't notice as i was allowing quite a large NTP pool at that time when testing. So the device was able to sync through NTP, just before it tried the fallback http syncing method.
'Great' to hear bootloop is narrowed down to a bug in http syncing method and only under certain conditions. And the other small issue of not using user defined NTP during boot, had yet to be fixed.
When people allow all outgoing 123 udp connections and/or http connections, bug will probably almost never occur because they will succesfully sync with a NTP or within early attempts of HTTP fallback method before the bug occurs. So the majority will never experience the bug :)
Thanks for all your work and sorry for the confusion i may have caused about the FW versions.
When looking at the connection flow, it's indeed happening after several failed http polls of worldtimeapi.
During boot routine, first it tries 5 x to connect to a NTP server, then fallback to http syncing. After several failed http sync attempts (10 - 20x) ,the bug occurs and the device reboots and repeats the procedure. Until it's able to either sync with a NTP server or by http worldtimeapi, then the bootloop stops immediately.
I stated this was not happening in 31 and before, but the problem is also present in earlier FW versions. I just didn't notice as i was allowing quite a large NTP pool at that time when testing. So the device was able to sync through NTP, just before it tried the fallback http syncing method.
'Great' to hear bootloop is narrowed down to a bug in http syncing method and only under certain conditions. And the other small issue of not using user defined NTP during boot, had yet to be fixed.
When people allow all outgoing 123 udp connections and/or http connections, bug will probably almost never occur because they will succesfully sync with a NTP or within early attempts of HTTP fallback method before the bug occurs. So the majority will never experience the bug :)
Thanks for all your work and sorry for the confusion i may have caused about the FW versions.
It's true as your analaisis, you are very good at it, if udp NTP method fails, will try the http method then. The small issue about user defined NTP server sync problem should be fixed since 3.2.37, but never mind, you could check the 3.2.39, I tested here, it will try the user defined NTP server once saved.
I have made some improvements about the http sync, but I can't block some website in my enviroment, hope you could help test about http sync crash problem.
Thank you very much for your time.
Looks good for now. No reboot since 20 minutes. NTP is also working but wrong timezone. I see some weatherapi calls but they are blocked and no further attempts are made when I do not interact with the device itself.
The bootloop is indeed gone. 5x NTP from non defined pool/server, then It keeps polling worldtimeapi endlessly every 3 seconds without the device rebooting. Very happy that the reboot issue has been resolved.
But....
NTP not using the user defined one. (during reboot) When you click save in the NTP interface , yes it will poll the defined NTP server once. But when rebooting, it won't poll the user defined NTP. It uses 5 random servers from some of the pools that are defined in the device. Timezone is also incorrect now. It shows Seoul time.
Crypto ticker interface is no longer working. It polls the binance api but not showing any visual data on the device. BTC 0.00% 1D , no price.
Firmware upload interface looks different (firmware and filesystem update in plain text)
39 FW seems to broke some things but fixed the reboot issue.
*** Seems like my device is a bit bricked now. I downgraded to 38 and did a reset to factory settings.
What i noticed is that my boot picture is no longer being loaded during boot. When i go to the 'picture' interface no pictures are visible. It shows a message : "failed to open dir" Downgrading / upgrading doesn't seem to fix this particular problem.
The bootloop is indeed gone. 5x NTP from non defined pool/server, then It keeps polling worldtimeapi endlessly every 3 seconds without the device rebooting. Very happy that the reboot issue has been resolved.
But....
NTP not using the user defined one. (during reboot) When you click save in the NTP interface , yes it will poll the defined NTP server once. But when rebooting, it won't poll the user defined NTP. It uses 5 random servers from some of the pools that are defined in the device. Timezone is also incorrect now. It shows Seoul time.
Crypto ticker interface is no longer working. It polls the binance api but not showing any visual data on the device. BTC 0.00% 1D , no price.
Firmware upload interface looks different (firmware and filesystem update in plain text)
39 FW seems to broke some things but fixed the reboot issue.
Thank you, that's good news about the boot loop. I have find out some problems after upload the 39 version to you.
1, Bitcoin App [confirmed]. 2, Check the user defined NTP server during boot. [ I need to check] 3, Restore the update web page. [confirmed]
Gif files could be downloaded here.
I've tried setting the weather info and noticed that the time switched 12hr notation to 24 hour notation, but timezone remains Korean time no matter what i do.
I tried clear before uploading and also reload doesn't make a difference. I'm unable to upload any png/jpg/gif. Free space remains the same. 'Failed to open dir message' remains the same. Guess i'm having some issue with the filesystem on that particular device.
- I've tried setting the weather info and noticed that the time switched 12hr notation to 24 hour notation, but timezone remains Korean time no matter what i do.
- I tried clear before uploading and also reload doesn't make a difference. I'm unable to upload any png/jpg/gif. Free space remains the same. 'Failed to open dir message' remains the same. Guess i'm having some issue with the filesystem on that particular device.
4/5 the reason are similar, parameters failed to save on the system. Did you upload the filesystem files on the update page?
- I've tried setting the weather info and noticed that the time switched 12hr notation to 24 hour notation, but timezone remains Korean time no matter what i do.
- I tried clear before uploading and also reload doesn't make a difference. I'm unable to upload any png/jpg/gif. Free space remains the same. 'Failed to open dir message' remains the same. Guess i'm having some issue with the filesystem on that particular device.
4/5 the reason are similar, parameters failed to save on the system. Did you upload the filesystem files on the update page?
Not that i am aware of, i uploaded the fw using the 'update firmware' button.
I noticed this update layout in earlier FW versions, but didn't touch it, as i expect FW should not be uploaded using Filesystem.
- I've tried setting the weather info and noticed that the time switched 12hr notation to 24 hour notation, but timezone remains Korean time no matter what i do.
- I tried clear before uploading and also reload doesn't make a difference. I'm unable to upload any png/jpg/gif. Free space remains the same. 'Failed to open dir message' remains the same. Guess i'm having some issue with the filesystem on that particular device.
4/5 the reason are similar, parameters failed to save on the system. Did you upload the filesystem files on the update page?
Not that i am aware of, i uploaded the fw using the 'update firmware' button.
I noticed this update layout in earlier FW versions, but didn't touch it, as i expect FW should not be uploaded using Filesystem.
OK.
1 Can you use chrome browser, and press F12 to enter web debug mode, click the network tab, and refresh the web, and then check if there's any failed links?
2 Confirm if you press save on the web, does the response success?
The device that has the filesystem issues :
https://github.com/GeekMagicClock/smalltv-pro/assets/75391786/a3f658a9-9350-45cc-a9c0-b5b1d9bf1187
Another of my devices that is working properly:
https://github.com/GeekMagicClock/smalltv-pro/assets/75391786/943bf3f6-f340-4dcf-8af7-435d4e86456a
Thanks for the videos. The fail video has input filters, I can't see if other requests failed too, can you help check if the autoplay save failed too ? I can see the image upload fail, 500 (Internal server error) returned by the device, this caused by fail to open file.
Thanks for the videos. The fail video has input filters, I can't see if other requests failed too, can you help check if the autoplay save failed too ? I can see the image upload fail, 500 (Internal server error) returned by the device, this caused by fail to open file.
You mean like this ?
https://github.com/GeekMagicClock/smalltv-pro/assets/75391786/b8a7a652-5a8d-4f1c-8573-1fde416432d3
Yes, can this file be visited? http://your_clock_ip/.sys/city.json, if you change the city in the weather option, this file should be changed, if not, the whole filesystem maybe damaged.
I can succesfully visit/access that JSON file at /.sys/city.json.
When i change the city code in the weather interface, it does change the city.json file after saving.
But the weather interface showing an 404 error for .sys/key.json when looking at debug mode.
Maybe it's better to open a new issue for this specific filesystem matter ? It seems only 1 of my devices has this problem, It seems to have happened after a lot of upgrading/downgrading actions, but i can't be completely sure when and how it happened, as i found out by accident.
Ontopic :
The bootloop issue seems fixed in 39 but the following issues are present.
Today i received a new Block/Clock so i can test from scratch with a fresh device. :) It currently has firmware .28EN
Maybe it's better to open a new issue for this specific filesystem matter ? It seems only 1 of my devices has this problem, It seems to have happened after a lot of upgrading/downgrading actions, but i can't be completely sure when and how it happened, as i found out by accident.
Ontopic :
The bootloop issue seems fixed in 39 but the following issues are present.
- Incorrect timezone. (Korean timezone)
- Manual NTP server not being using during boot.
- Bitcoin feature not working anymore.
Today i received a new Block/Clock so i can test from scratch with a fresh device. :) It currently has firmware .28EN
Yes, it's better to create a new issue to track the filesystem matter.
FW_SmallTV-PRO_V3.2.40EN.zip Here's change in the new version:
Use the user defined NTP server during boot.
Incorrect time zone -> The timezone will be updated after the weather update, and saved to the filesystem, I can't duplicate this on my device, is the weather info correct ?
I can succesfully visit/access that JSON file at /.sys/city.json.
When i change the city code in the weather interface, it does change the city.json file after saving.
But the weather interface showing an 404 error for .sys/key.json when looking at debug mode.
Seems only the folder /image is damaged, can you check if there're gifs under the weather option? It is under the /image/80x80.gif/ folder, may be disappeared ?
FW_SmallTV-PRO_V3.2.40EN.zip Here's change in the new version:
- Restore the Bitcoin App.
- Fix Stock App price change percent.
- Use the user defined NTP server during boot.
Incorrect time zone -> The timezone will be updated after the weather update, and saved to the filesystem, I can't duplicate this on my device, is the weather info correct ?
3.2.40EN has resolved all issues.
Boot issue is fixed. (I tried with blocking all connections)
The device is now utilizing the user defined NTP during boot (and uses the devices pools when no manual server is entered.) When udp 123 is not available the device will fallback to http time sync. It won't crash no more.
Time syncing issue is resolved. Going to the weather (management) interface and save the city info, did not do anything (no connections except polling NTP / HTTP worldtimeapi and if entered the user defined NTP) But.... when i actually enter the weather info screen on the device, it polls openweathermap IP. When i allow it (just once) it corrects the timezone. After this action it remembers the timezone and uses NTP / HTTP sync / Manual NTP server to correctly sync the time. When i manually reboot the device, timezone is remembered and time is correctly synced.
I've tested this succesfully on my device that was not properly working. (filesystem issue) Great work! Very happy with your support. 🥇
Will open a separate issue for the filesystem issue, but not really a problem if it can't be fixed. The image map indeed seems to be missing or maybe is corrupted.
Thanks again for all your hard work! (i will finally close this issue)
I'm very happy to hear that, thank you very much for your testing and time spent on this, I'll release the V3.2.40EN soon.
After updating to V3.2.35EN , my devices keep rebooting after a 5 - 10 seconds.
Luckily i was able to downgrade it to 3.2.31EN, but i guess there is quite a risk, people might brick the device, when the device reboots while updating the firmware.