MAC Stops Changing

[KamiYamabushi]

I've noticed if I leave PiPass 1.2 running for over several hours, the MACs will eventually stop changing despite having a lot more in the list to go through on shuffle (Extended MACs list).

I currently have the refresh time set to every 6 minutes and it hasn't changed in over 30 minutes now. This is not the first time this issue occurred.

Additionally, sometimes starting the PiPass service, restarting it, or even trying to shut it down does not seem to function properly. Typically requires a full restart of the Raspberry Pi 2 instead.

[Matthew-Hsu]

I'll comment about the starting, restarting, or the shutting down part first. You may have seen this with changing the settings, but sometimes the browser will cache the web page with the older values. Often, refreshing the web page will force the browser to pull the new values to show. I will probably look into forcing a reload with these pages automatically, but stopping the service should work all the time. I will normally run piPass.py manually through terminal to get printed logs and if I stop the service through the dashboard, you will see it get stopped. I'm guessing the browser cache is not reflecting the correct status without a manual refresh.

I know sometimes that starting the service, stopping, and then starting again will have some issues. Hostapd generally needs some time in between restarts. I would say 10 seconds between a stop and a restart would be enough to allow it to fully stop.

As for the extended MAC changes, I will look into it and let it run overnight on the extended MACS to see what is going on.

Thanks for the feedback and I'll try to see if I can replicate and fix the issue.

[trowgundam]

I have seen this exact same thing. For example, I currently use the default list of MAC's and I have the shuffle turned off. I got through about 2/3 of the list and it stopped changing between access points. My interval was set at 9 minutes (so it would take roughly 8 hours to go through the whole list).

I also see the other issue. Just now I tried restarting the Raspberry Pi through the web interface and it did nothing. I had to power cycle it in order restart. Also when it came back up I had to hit the Start button at least 3 times (waiting about 30 seconds to see if it would go up) before it actually started. I have seen the Settings page stick with a cached version, but that can be easily fixed with a Forced Refresh (Ctrl-F5) and didn't really bother me too much. In Internet Explorer this doesn't seem to be a problem for some reason.

[Matthew-Hsu]

@KamiYamabushi @pokeguru87

I've updated the master branch with a new potential future version. @nagledb has contributed some pretty nice changes that does change some logic on the list traversal. Not sure if this rework could have fixed this issue as a side effect.

I did replicate this issue a few days ago. It does seem intermittent and it seems to only occur over a long period of time. I did set the cycle time to 1 minute and PiPass is able to traverse the entire list fine, so I do not think there is something wrong with traversing the list. It could be something with Hostapd and I can experiment with some things with it to see if it makes a difference. In the meantime, if you want, could you try updating PiPass to the master branch version? You just need to replace the files in /opt/PiPass/ and the files in /var/www/ with the ones in master.

I've also included some changes regarding the caching, but it seems a little shaky on the client. It will force an update occasionally, but this may be something that I have to configure on Apache to get it consistent.

Thanks!

[KamiYamabushi]

I've updated, will leave it running during the day while I'm at work to see if it still hiccups.

Thanks!

Edit: After over eight hours, I came back to a stuck MAC again. The issue is still there even with the update. Perhaps a timed stop/restart of service built-in at hourly intervals (while keeping track of the current list) might be helpful?

Will leave it running to see if it happens again.

[nagledb]

I've encountered this behavior several times.

It just happened for me again a few minutes ago. Watching the dashboard status, it looked like my piPass was running but stuck on the current zone. But when I ssh'd into the system, "ps aux | grep piPass" showed that piPass wasn't actually running. I'm guessing that it crashed at some point and when it did, it left enough state behind that the dashboard doesn't realize it's dead.

I also have noticed that it struggles to start back up after this happens. In this case, I tried starting it back up from the dashboard several times and it didn't work. It finally tried running it on the command line over ssh and then it worked. I CTRL-C'd that instance and then was able to start it back up from the dashboard after another try or two. So maybe piPass is crashing and leaving behind an inconsistent state that is causing problems when it tries to restart?

Nothing in the code jumps out at me as an obvious candidate for causing this kind of problem, though.

[Matthew-Hsu]

Just a quick response on this. I've been a little bit busy this week and won't have too much time until the weekend, but I have ran PiPass manually through piPass.py and it has been running for over 8 hours without any problems. If you run it this way, you can still control it through the dashboard.

I will probably run this longer and on new instances of it, but it could be possibly related to piPassCommand.py as it calls piPass.py in it. If it seems to only crash when started through piPassCommand.py, then I will do some rework and test that out on a testing branch.

[trowgundam]

Not sure if it helps, but I wrote a simple little desktop application that will monitor the Current State of PiPass (same as the Dashboard of the web interface). It also keeps a log of things as it detects them changing. The thing I think makes it the most useful is it sends the the piPass.py command through an SSH connection. It is by no means complete, and it could still have bugs. But in my testing it works to monitor things and can start and stop PiPass. I had a few ideas on additional features, like auto restarting the PiPass if it was idle too long among other possible things. Again, at the moment it works for what I originally intended, but not much else. I could expand the functionality, but only if there is interest for such a utility. Here is the link: https://www.dropbox.com/s/fkxx7oe7gjsuhcw/PiPass%20Monitor.zip?dl=0

You'll need to go to the file menu and set the IP address of the RaspberryPi. It defaults to refreshing every 60 seconds, in my testing I used a 1 second Refresh Interval and it worked just fine. Realistically, I think leaving at 60 seconds is fine, but 15-30 seconds should be just fine without hammering the HTTP on the RaspberryPi. If the process errors out it will stop the refresh cycle and you will have to go in and change a setting the properties (add or remove a second from the Refresh Interval, one of the settings just has to change for it to revalidate the connection). You could also just restart the application and it will attempt to reconnect to the RaspberryPi.

[Matthew-Hsu]

It's pretty awesome to see that you've made a desktop application that interfaces with PiPass and is a better fit for what you wanted out of it.

I'm currently testing some changes and fixing some design decisions locally on my end to just have PiPass "flow" better behind the scenes. I've been mostly collecting information and I hopefully will have a fix for this issue soon as long as all these changes pass long duration tests. If not, I do think your method of monitoring could be helpful here.

[trowgundam]

Well my app has shown me one thing. Just sending the "sudo python /opt/PiPass/piPass.py" command over SSH didn't solve the issue.It still stopped swapping at the same Hotspot for me. However, when I login through SSH using PuTTY and run the command, keeping the session open, it gets pass it just fine. So, that makes me think the reason why it didn't work for my application was because I killed the SSH session after I sent the command. I don't know why it would act that way, but my guess is it is a timing thing since it always stops on the same Hotspot for me (I have the shuffle turned off). Maybe the system sees the process and kills it after a set time if the session that invoked it is no longer active. With one of my test runs, I started the script at 3:15:10 PM and the last Hotspot became active at 8:57:49 PM, so it is right around the 5.5 hour mark that things go wrong. Perhaps the script should changed so that it runs only to change the access point and then just schedules itself to run when the next change is done. I'm not sure how to go about this, as I'm not terribly familiar with Linux (enough to get by, but not to program anything for it). In Windows it would just be setting the script to run as a scheduled task and the script just changed the SSID and MAC address to the next one when run, instead of staying running.

[Matthew-Hsu]

Yes, I suspected it was a session issue. I made some changes so that www-data is now executing piPass.py whereas it was executing piPassCommand.py before and then piPassCommand.py would call piPass.py. Since www-data belongs to Apache, it should hopefully run for the duration of time now.

I believe doing it that way caused it to end prematurely. I ran PiPass overnight and it is still running. I will double check after work and if it is fixed, I will push these changes to master and probably do an official release shortly after.

Fingers crossed! Thanks for the feedback, at least we understand the problem now.

[nagledb]

Won't the issue still remain though if you start piPass manually on the command line via piPassCommand?

Googling around, it sounds like if you have piPassCommand do a double fork, it should properly daemonize the process. Some links on that topic:

https://stackoverflow.com/questions/881388/what-is-the-reason-for-performing-a-double-fork-when-creating-a-daemon https://stackoverflow.com/questions/1417631/python-code-to-daemonize-a-process http://www.jejik.com/articles/2007/02/a_simple_unix_linux_daemon_in_python/

Or possibly better yet, there's a library for it available through pip: https://pypi.python.org/pypi/python-daemon/

[Matthew-Hsu]

I closed my laptop before I left for work, so I cannot remote in and check the current status, but it was still running after that 5.5 hour mark that was identified. It traversed the entire Nintendo Zone list and started a new iteration of the list.

I added a additional signal and PiPass has a proper terminating state now. There is no more killing of processes and the need of piPassCommand is gone now. www-data should have full ownership of the PiPass process and I hope the ending bug has been fixed with these changes.

I will try to provide an update with what happened sometime tonight.

[Matthew-Hsu]

I've pushed my changes to master that should fix the issue. I had PiPass running for almost 24 hours without any hiccups. I've removed the need of piPassCommand now.

[trowgundam]

I've updated my device, and things seem to be working just fine. I'm letting it run its course while I'm at work, and I will verify it is still working when I return home and made it past the normal point it would stop. However, one thing I did notice is that calling the "start.php" page takes considerably longer now. On the web page it takes a while and the HTTP server doesn't respond for awhile. I saw this by after hitting the start button on the web interface and then trying to go back to the Dashboard and I actually got a flash of the Chrome "Connection Timeout" page before the dashboard finally loaded. I also noticed this once I changed my application back to calling the "start.php" instead of invoking the process through SSH. It would lock up the application for a good 15-20 seconds (I was doing things on the UI thread because I was lazy). I just moved things to a background thread for my application, but the delay is still there. One thing it did tell me is that the access point starts up almost immediately, it is just the call to "start.php" that takes a while to complete. To be honest, if it fixes the issue, I'd consider it a mild annoyance instead of bug and probably wouldn't bother me since I wouldn't have to restart the process every so often. I've also removed the SSH reference and modified a few things in my app. For those that want it, here is the link to the updated version: https://www.dropbox.com/s/fkxx7oe7gjsuhcw/PiPass%20Monitor.zip?dl=0

[Matthew-Hsu]

I believe I did some unnecessary Hostapd stoppage checks. Sometimes, I like being explicit, but looking over some of the PHP files, I don't necessarily need to touch Hostapd, especially during start.php. piPass.py will take care of that regardless.

I've pushed a minor change that should give you the same performance with the previous version.

[Corruptinglyneedful]

I can't wait until you get the ofifical update out. It is breaking quite consistently for me. Not even sure time intervals, just know I've restarted PiPass over 6 times today.

How does it update, anyways?

[Matthew-Hsu]

If you're using the exact same kit as me, which you stated before, that means you've been restarting manually every 5 to 6 hours since yesterday for over 6 times? Because this sounds like an issue that is unrelated to this since it sounds like it is breaking very fast.

I don't have much time to give you a step by step instruction set right now, but I often use FileZilla. Try Googling it. The most important information you need to know is:

Host: Pi's IP Username: root Password: PiPass Port: 22

Did you read the previous response from the last thread you asked? I recommend checking that out since you will have to delete /opt/PiPass/ and /var/www/ on your Pi. Easiest way is to use FileZilla for the entire process.

On the main GitHub page, on your right, there is something called Download Zip. Make sure you have master branch selected. Download that and extract it. You should see folders /opt/ and /var/. If you go into those, you'll see the folders that you just deleted on your Pi (PiPass and www). So, copy those folders over to your Pi.

In the end, you should /opt/PiPass/ and /var/www/ again on your Pi.

[Corruptinglyneedful]

That's greek to me. I have not a clue what is being talked about there. Perhaps someone can offer a super-dumbed down explanation.

There clearly isn't a patch button or an updater to update PiPass, which is what I was looking for.

Soo, I'm guessing you have to reflash the image every time you update, which means re-configuring all the time.

Is that correct?

[Matthew-Hsu]

FileZilla is basically an application that looks a lot like Windows Explorer in where you can just drag and drop files. Basically, it allows you to copy over files from your computer to your Raspberry Pi. If you do this correctly, you can update PiPass in under 1 minute.

You did make a good suggestion about updating, so I've implemented a update button into PiPass where it will do all of this automatically for you. This will preserve your settings as well.

You'll have to be a little patient and I will update another image with all these changes. If you really want these changes now, download FileZilla from here: https://filezilla-project.org/. Install it and re-read what I posted above. It should make a lot more sense when you actually see it.

[Corruptinglyneedful]

You, I like you.

Glad to hear updates now preserve settings and is even easier. Should help attract more people to the platform.

I think I vaguely understand how FileZilla works, just curious how it can update PiPass when it's running?

You had mentioned something about Sunday being able to implement the Mac Cycle fix, yah? So will the update feature be on it as well?

I really can not wait for the mac cycling to be fixed. I've had to restart it 2 times today, already. I don't know why yours is lasting 6 hours, but that's not my experience and yes I do have the same kit as you.

I plugged everything into the Pi, flash PIpass, added the mac adderess for my 3DS, configured my settings, and booted everything.

I did nothing else.

Thank you for being so helpful Matthew.

[Matthew-Hsu]

I can try to help you out. I'm going out tonight, but I can give you the exact commands to upgrade PiPass. All you need to do is be able log into your Pi and run terminal commands.

All the features will be in the next release. The MAC bug fix works for me. I have not confirmed it with others, so you can help with the testing.

Please stop PiPass and run these commands in order, do not skip a step.

1. sudo apt-get install p7zip-full -y
2. sudo wget -P /tmp/PiPass/ https://github.com/Matthew-Hsu/PiPass/archive/master.zip
3. sudo 7z x /tmp/PiPass/master.zip -o/tmp/PiPass/ -y
4. sudo cp /var/www/assets/json/pipass_config.json /tmp/PiPass/
5. sudo rm -rf /opt/PiPass/
6. sudo rm -rf /var/www/
7. sudo cp -ar /tmp/PiPass/PiPass-master/opt/PiPass/ /opt/
8. sudo cp -ar /tmp/PiPass/PiPass-master/var/www/ /var/
9. sudo cp /tmp/PiPass/pipass_config.json /var/www/assets/json/
10. sudo chmod -R 755 /opt/PiPass/
11. sudo chmod -R 755 /var/www/
12. sudo rm -rf /tmp/PiPass/

If everything goes well, you should have PiPass upgraded. If it does not work, make sure you type these commands exactly. If you reach the end and it still does not work, start the process again.

You'll know if you have upgraded correctly if you see the 'update' feature under PiPass.

Good luck.

[Corruptinglyneedful]

Holy commands, Batman. So that's how you do it without using FileZilla.

So how does the Updater do it?

[Corruptinglyneedful]

I ran into issues.

I had to re-flash the MSD card, it got bunged up.

1. I could not get the commands to work, it wouldn't update. I entered them correctly several times.
2. I got the update to work with Filz Zilla, I could see Update under Dashboard, but File Zilla had an error on one of the transfers.
3. When I used the Update on the Dashboard, it broke everything and I had re-flash the MSD card.

I'm now back on 1.2

[Matthew-Hsu]

I forgot to update you. Those FileZilla instructions were only good for that MAC bug fix. With the updater, PiPass needs something installed before you can update that way. The commands cover this and will work.

It's not official and things may change, but visit the download links. 1.3 is out now if you want to upgrade. I'm just letting PiPass run longer to double check that some new changes didn't break anything. I believe it should be fine.

[Corruptinglyneedful]

Where is the disc image for Version 1.3?

[Matthew-Hsu]

https://github.com/Matthew-Hsu/PiPass

Documentation hasn't been updated, since nothing is official. But, it's where you downloaded 1.2 on Google Drive or Dropbox.

[Corruptinglyneedful]

@Matthew-Hsu Thank you!

I'll image the new file and keep an eye out for bugs.

So this image contains both the mac cycling bug and the updater in the Dashboard?

[Matthew-Hsu]

Yes, it contains both the fixes and other stuff as well. With the MAC bug fix, I ran it for 24 hours and it worked. I did make some minor changes that shouldn't break that fix, but I am just retesting to make sure. So far, it's been 5 hours running and is still cycling MACs.

[Corruptinglyneedful]

I am downloading since 11 minutes ago.

So if you make any changes or discover something please tell me, as I'm about to flash the card.

[nagledb]

I've been running the recent changes and things seem to be working well for me now. No need for manual restarts so far.

[Corruptinglyneedful]

I'm writing the card image, right now. So I'll get to join this club shortly.

[Corruptinglyneedful]

What is this "Advance" feature for?

[trowgundam]

I've had it running over 24 hours (originally started from the Web Interface) and everything looks to be going just fine.

Corruptinglyneedful: The "Advance" will force the script to move on to the next Access Point in the list, instead of waiting for the time limit. You could use it to just go through the whole list as fast as you could if you wanted. Not that you would have time to use the street passes gained since most games cap it rather low (most are 10, Pokemon games are 30 and a rare few are the max of 99 street passes).

[Matthew-Hsu]

@nagledb has implemented a nice feature where if you like having your cycle times at a higher value, you are given the option to force a move to the next Nintendo Zone in the list. It's a nice addition if you want more control in your StreetPasses. Of course, PiPass will still auto cycle regardless of this.

It's been more than 8 hours and I have not seen any issues with the MAC bug. @pokeguru87 and @nagledb have not reported any new issues, so I am going to close this issue. If you have specific questions, please file a new issue to keep things more organized here. The README has been updated with a complete list of changes.

To everyone, thanks for all the contributions and feedback. It was very helpful! :dancer:

[Corruptinglyneedful]

It has been over 17 hours cycling and it has not had to be manually changed. Thanks everyone.