Reduce grabber delay

[saithis]

To reduce the delay till the task is started, you could check in the goCAST.php if a task is is running and if not, just start the new task without saving it into the mysql Items table.

[lanceseidman]

Only reason why that isn't likely to happen? If they want to change videos or load a different website, play another song, etc it wont work while the current task is active.

[saithis]

Oh, is the code currently killing old tasks and executing the new one if a new one is found? If so, you could also just skip the mysql db completelly and do that directly after receiving the new task in goCAST.php.

I didn't read the code closelly enough and tought the mysql db was like a playlist :S

[InfernoZeus]

As far as I understand it, goCAST.php stores the information into the database, and every 5 seconds grabber.sh polls the database using grabCAST.php and executes accordingly. I'm not sure why grabber.sh can't access the database directly..

[saithis]

Yes, thats also how I understand it (now), but why saving it in the database in the first place? Just execute the command as soon as you receive the task. The database would only make sense, if you use it as a playlist.

[InfernoZeus]

Agreed.

[lanceseidman]

The reason grabber doesn't constantly make local SQL Connections and process everything on the fly is eventually it will get too big. But more important, everything is supposed to be processed via web as with the current method you can request/process requests to run by, remotely.

Why is that important? If you can't keep a home server running due to your ISP (they block and forbid running your own server) you can put it on another hosting service. Why not remotely connect to that SQL DB then? As a Web Host, I don't permit remote connections to my clients SQL DB's as it's a possible threat. Could you white-list an IP? Sure but now you're going way over the normal user.

The current setup allows you to run it local or receive the commands remotely with a defined URL in which everything is processed and ready to run and if one chose to have grab sh as a service, it's not big and strictly just makes a tiny GET request with literally almost 0 processing. Have to remember the Pi isn't so powerful., every browser window takes a lot out of the Pi and we can process the actions faster if PHP is handling what should be done in the background. Maybe not much at this specific moment, but it will add up but shortly once the other features start rolling in and it'll be easier to add/customize new features via PHP with a web-based control panel.

Beyond all of that, soon the other reason is we want to be able to allow the user to send multiple requests or have a list like functionality and eventually "store" them as they come in and as processed to remove itself. The idea is you'll be able to have specific items load at a certain time/date and eventually even daily/weekly/monthly. Of course you could never have known any of the above but I promise in the end it'll make sense but if it doesn't, I hope you'll tell me. Just wait a little bit longer with me.

InfernoZeus: The grabber works as fast as you want, no defined speed. When you setup the watch you select the interval. So it can be every second to 8 or 30.

[InfernoZeus]

@lanceseidman I was aware you could change the interval, just didn't mention it as it didn't directly affect my explanation.

It looks like you've got a lot of extra features planned, so I'll wait for more info before commenting on them. :)

[lanceseidman]

Sounds good, just had to mention it. Because in my video I used 8 seconds and have received crap for it, "that's too slow! Shut this crap down!"... Where as you're obviously informed the number means nothing.

If eventually it doesn't all make sense, I hope you're the first to call me out on it!

[saithis]

Ok, now the grabber makes sense to me, the only 2 things that bug me are:

1. When all the logic is on the webserver and the pi only grabs the commands, then a playlist is a feature that will never work, as the webserver doesn't know when something is done playing.
2. Streaming files from my local pc won't be possible (the feature I like the most on Chromecast)

[lanceseidman]

Re: 1&2). Have you seen Webmin before? Via browser you can see in real time what processes are running on your system (as well as a junk load of other items). For Playlists we can use JSON to grab video ID's from YT videos desired to play and add a minor delay but in SQL grab the ID, go fetch details and force site refresh at specified video length+delay and then process and keep doing it over and over, another reason to love MySQL (besides it will store your YouTube API+Key & other settings).

I mention Webmin though, because we can do the same thing and that's the idea of a web control panel I am envisioning where "modules" or "apps" to make people happy that can do things and like Wordpress allow you to type for a app and download/install or upload your own.

Besides the upload your own App idea, we can use that if people want to store media on the Pi OR what others are doing with the Chromecast and enable Box/DropBox media playing (using the cloud to play media) by using the those companies API Calls or visiting the webpage to display on the Chromecast/TV.

Chromecast is only so advanced because they're using a Browser that's been worked on for years and I imagine more than 6 days on the Chromecast & its functionalities' with a big dedicated team. Can't really expect all of this to be a decent solution yet, already it does 2 main things: YouTube video & website launching w/Mouse Control for Windows, OSX, or Linux.

Eventually everything will be right, just not yet. But Images are about done, using PHP to use Base64 String conversion for images and another reason MySQL is great, we can use BLOBS.

In the end, it'll still never be 100%.

[saithis]

For this all to work, the pi has to be accessable from the internet and the webserver gives the pi console commands to execute. This is the best things you can do, if you want to get hacked. Yes there are solutions to make this a bit more secure(using ssl, etc.), but many of them can't be automated and so inexperienced users will just run it without making it secure. I don't want to offend you, but if you plan to support webservers in the internet, then making it secure will get really hard.

Also I can't think of a reason why you would need a external webserver. If you are not in the same network as the pi, you are most likely not at home. So why would you want to start a video on it without being able to watch it then? Your ISP can block/forbid running servers accessible from the internet, but if it isn't exposed to the internet, your ISP can't do anything about it. Or am I missing something here?

I don't expect PiCast to be a decent solution yet, I'm just saying you should consider the planned features in your design now, so you won't get in trouble later. Also maybe you should decide which features you want to support and which you don't care about and write it down somewhere, so if someone wants to contribute, he knows what to work on.

[lanceseidman]

The PiCAST sets up a LOCAL Web server. It can be accessed by any machine locally on your subnet. This means 192.168.x.x can immediately access it. As most ISPS block Port 80, it wont be accessible by default and most routers today you'd need to setup forwarding and in some cases DMZ for a successful WAN connection.

As with the installer (constantly being changed), it copies the basics from cloning the GIT then throws it in the /var/www & mention because you can change the directory name and now someone looking for it, wont find it. In which you can easily setup in apache config or setup IP Tables if you're paranoid for a white list or even just a C-Class allowed only list but also can change even the Port # (obviously) and unless someone was sniffing/scanning your IP-- likely wouldn't find it.

But also authentication will be needed, eventually. Not worried about it at this moment, most who have a RasPi aren't idiots but that's where SQL will come in later & with each request throw the auth w/it and for those still paranoid can go make a salt/hash script or OAuth.

For the remote loading of files/video, if your Pi isn't on the same network as your Phone, Laptop, etc & as you mention users may not be very tech savvy, having a web control panel can house settings, one could even be a temp. unlock of blocking traffic & guest password (expire maybe in 3 Mins, w/e). So if you're doing a presentation & video needed or using a TV w/WiFi & Ethernet input (obviously on a different network than perhaps your phone, maybe not). At my home my switch (Pi connected to) and my router (WiFi connected to) are separate IP subnets (192.168.0.x/192.168.2.x). So a "remote" connection is what I need if I blocked the IP subnet to be only X.x.0.. as I would be in X.x.2..

As for writing a list? The whole idea for me making it open source and public (opposed to never releasing it) is if someone has an idea that can addon to make a better feature or a new feature, do it make it and show it off. If they have something working, lets see it and if its open code & willing to share w/everyone, lets add on to the project. I don't really want to say what I am doing and someone thinks oh I won't bother cause he or anyone is already doing that. Forget that, likely they could have something 100% better than 5 others trying to write the same thing OR parts of said persons code is good and others bad but lets join it to guy/girl 2's code.

Lets be real though... In the end, it is public, someone doesn't like how I am proceeding, 100% ok. They're always welcome to build based on what already exists by forking the project & never deal with me or any contributors, it's likely they want a private solution, I know I originally did & figured others may too, so again I am just sharing. It may not always be right or the best way (yet or ever) but whoever is welcome to do better, which I think is the point of a community project, get big thinkers like yourself & do something for the hell of it.

[saithis]

Scroll a bit up and reread your own post, there you wrote "If you can't keep a home server running due to your ISP (they block and forbid running your own server) you can put it on another hosting service.", thats what I was refering to.

Edit: Oh, and you wrote, that "Pi isn't so powerful" and a external webserver should handle the load.

[lanceseidman]

In that instance, if you didn't want to run a local Pi server (assuming you wanted to do the remote thing), you can change the grabber binary and change from localhost/goGRAB.php to remote-website.ext/goGRAB.php. You really don't even need a Web Sever, if you could just write to a file you can request it with the grabber.sh.

I am using Apache, PHP, and SQL for other features to come or whatever else someone wants to write.