search

bren1818 / TCPLightingWebInterface

Web App interface to control the "Connected by TCP" bulbs
MIT License
30 stars 15 forks source link

Comments, Questions, open forum (2) #6

Open bren1818 opened 7 years ago

bren1818 commented 7 years ago

Closed the last topic as it had gotten really lengthy. Opening this one for more questions

sktaylortrash commented 6 years ago

@bren1818 The single usb port is probably the biggest downside to the Pi Zero during setup. I usually use a hub and independent keyboard and mouse. I'm guessing that until the system is fully loaded it doesn't have access to the full HID driver set to load what it needs for combined devices.

I didn't include things like no-ip or let's encrypt mainly because they're not applicable to everyone, or necessary to get up and running. For example I have static IP's and own lots of domains

If you are interested and want to enable the wiki function and give me permissions. I'd be happy to start building out documentation for the project. Might help to lower the entry barrier for new users. There's a lot of helpful data in this thread for example but it's a long read to find it all.

bren1818 commented 6 years ago

@sktaylortrash - Great idea on wiki! I have much renewed interest in this project. Appreciate you reaching out. I started a wiki and invited you as a collaborator to the project.

Cheers,

Bren

sktaylortrash commented 6 years ago

@bren1818 right on see you're already in the making some placeholders

bren1818 commented 6 years ago

@sktaylortrash Yessir 👍

I have to head out for the afternoon, but when Im back this evening I'll see how to grant you access to the Wiki.

Cheers

sktaylortrash commented 6 years ago

@bren1818 Look like adding me as a Collaborator was enough. I've added a simplified version of the Pi instructions. It eliminates the virtual host stuff and just has the barebones for setting up a server for the project. I think I'll do a second one that has the more advanced stuff or add it at the end as an option

merrickw commented 6 years ago

@sktaylortrash

I just tried out your guide, but I must be missing something.

I had Wifi connected during the NOOBS install, but now that I'm at the prompt I don't think wifi is enabled at all. I get no mention of wlan0 when I try ifconfig -a.

Also, I wonder if I should've loaded the full Raspbian instead of lite - since I'm new to this I wouldn't mind seeing the GUI ... is there a reason you recommended lite?

merrickw commented 6 years ago

@sktaylortrash just an update, I went into Recovery, back to Noobs, and am installing full Raspbian; that way I can hopefully see the wifi icon on the screen to configure ...

sktaylortrash commented 6 years ago

That's really weird on the Wi-Fi I did the install twice to be sure I had the steps down. But I did just do a quick google and apparently others also have the issue

I only recommended lite because the gui is really unneeded in a web server environment. It loads all kinds of programming tools and at at least at the quick glance I took had no tools for configuring web server stuff so that was still all going to be command line anyways.

merrickw commented 6 years ago

After installing the full graphical Raspbian, wifi is connected; I'll continue on later when I get back from a kids birthday party ...

sktaylortrash commented 6 years ago

weird but glad that's working. I've ordered myself the same kit in case my pi zero is a different version or something. Plus I'm sure I can find a use for it as something else afterwards

bren1818 commented 6 years ago

@merrickw re: the wifi not turning on - I had the same issue (or so I thought). I had installed and configured with the GUI and then disabled it and made it boot into the CLI. When it rebooted it didn't seem to connect. It did connect eventually, but it took about 20 minutes for some reason. I ended up ssh'ing in and having it start with the GUI and using VNC for future wifi toggling. Having the GUI on doesn't seem to slow it down any so long as nothing is open.

merrickw commented 6 years ago

Well, I pummeled my way through and got it all working. I installed the full graphical Raspbian and that left the wifi on. I had various problems along the way, like not having permissions to the html folder - I changed ownership to pi user, used chmod 777 (instead of 776 from the wiki - not sure if that made a difference). I also had a problem with Apache executing PHP, so I ran this: sudo apt-get install php libapache2-mod-php -y and that fixed the apache-php issue. I may not of followed every step in order, but it's working, headless, no monitor, mouse, or keyboard. I just use putty ssh to connect. Thanks for all the help guys!

anick08 commented 6 years ago

@bren1818 @sktaylortrash

Would there be any issue with running this on an existing Windows Home Server (But we have Win10 OS, not WinServer installed) instead of a RPi

bren1818 commented 6 years ago

No issues. I developed this on Windows 10 and use xampp as server. It should be a-ok. I just am using a pi so I don't have to leave a desktop on.

On Dec 4, 2017 6:40 PM, "anick08" notifications@github.com wrote:

@bren1818 https://github.com/bren1818 @sktaylortrash https://github.com/sktaylortrash

Would there be any issue with running this on an existing Windows Home Server (But we have Win10 OS, not WinServer installed) instead of a RPi

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/bren1818/TCPLightingWebInterface/issues/6#issuecomment-349144729, or mute the thread https://github.com/notifications/unsubscribe-auth/ACDCH8MrcrRLTRC5sboqbFN9fwD4H_Ceks5s9IL1gaJpZM4LEhw5 .

sktaylortrash commented 6 years ago

@merrickw Glad you are up and running - once I get my Zero I'll go through it all again. I'm just not understanding why you are seeing such different results. I definitely didn't need to change ownership of anything to get it work and 766 should give you read write to the folder and files

@anick08 If you want to run it with IIS the windows builtin webserver rather than xampp you'll need to look into installing fast-cgi, php and enabling php-curl and php-xml

anick08 commented 6 years ago

I don't mind using XAMPP, I had this up for a hot minute before using that, but began to not use the bulbs, with IFTT and Google Home, I am reinterested in this.

bren1818 commented 6 years ago

@anick08 Yep, voice control makes things more fun. I'm going to put some time into this project and re-write some of it. XAMPP works pretty nicely IMO.

bren1818 commented 6 years ago

@merrickw - I have a fix for the scenes issue. I'll be rolling out an update in the next few days... I'm over-hauling the UI and going to port the scenes into the main landing page, and their api calls to the api file to make everything more streamlined.

If you want to do a quick fix, modify scenes.php line 51 to: $CMD = "cmd=SceneRun&data=<gip><version>1</version><token>".TOKEN."</token><sid>".$sceneID."</sid></gip>"; essentially just remove the <val>1</val>. I'll work this into the release.

Cheers,

Bren

sktaylortrash commented 6 years ago

@bren1818 random thought. It seems that include.php has sort of grown over time to include a lot more functions than it did originally.

I'm wondering if for the sake of easier updating it might make sense to move all the user defined variables out to their own file. Preferably one that doesn't exist in it's final form on git. Like make a config.inc file necessary to run the program but in git have a file called config.inc.default. Then one could use git to download any new releases and not have their customized variables overwritten. The when a new variable needs to be defined it could be turned off by default unless the variable is defined in the config.inc file. To enable it the end user would need to read the commit comments or an updated readme. But at least downloading the newest version wouldn't break anything.

Hope that made sense

bren1818 commented 6 years ago

Haha that makes perfect sense. It's actually part of the over-haul I'm working on because I got sick of committing only parts of the include file. As you suggested, I kinda copied the phpMyAdmin approach of having a config.inc file.

Will keep you posted. Hope to push the changes in the next couple of days. I've also cleaned up the skin and will be moving some of info into the wiki.

On Mon, Dec 4, 2017 at 11:26 PM sktaylortrash notifications@github.com wrote:

@bren1818 https://github.com/bren1818 random thought. It seems that include.php has sort of grown over time to include a lot more functions than it did originally.

I'm wondering if for the sake of easier updating it might make sense to move all the user defined variables out to their own file. Preferably one that doesn't exist in it's final form on git. Like make a config.inc file necessary to run the program but in git have a file called config.inc.default. Then one could use git to download any new releases and not have their customized variables overwritten. The when a new variable needs to be defined it could be turned off by default unless the variable is defined in the config.inc file. To enable it the end user would need to read the commit comments or an updated readme. But at least downloading the newest version wouldn't break anything.

Hope that made sense

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/bren1818/TCPLightingWebInterface/issues/6#issuecomment-349191745, or mute the thread https://github.com/notifications/unsubscribe-auth/ACDCHwZb4Ixk7r2kidkceumhlwHzAlg5ks5s9MXzgaJpZM4LEhw5 .

sktaylortrash commented 6 years ago

Okay cool. On another note before I do more. How are you with the styling I've gone with on the Pi and DDNS entries in the wiki? Anything you'd like changed?

bren1818 commented 6 years ago

I like what I see and really appreciate you taking the time to do so. Also I'm loving the feedback and suggestions. I have a long bulleted list of things I want to do. Thanks again for your time and work! :)

On Tue, Dec 5, 2017 at 9:09 AM sktaylortrash notifications@github.com wrote:

Okay cool. On another note before I do more. How are you with the styling I've gone with on the Pi and DDNS entries in the wiki? Anything you'd like changed?

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/bren1818/TCPLightingWebInterface/issues/6#issuecomment-349314505, or mute the thread https://github.com/notifications/unsubscribe-auth/ACDCH3KYt2CAPZRfvTRd56IO11FAdrYWks5s9U6EgaJpZM4LEhw5 .

sktaylortrash commented 6 years ago

Not a problem glad to help in some way. It's been a long time since I did coding as a job so I'm only helpful conceptually. I'll hold off on integration and installation documentation until you get this next round of changes sorted

merrickw commented 6 years ago

@sktaylortrash , I did have some feedback on your original document if it helps ...

@bren1818 , I tested the scene change and it does work, thanks for the fix.

Now for a couple of questions... 1) Yesterday when I changed the hostname of the pi I had no problem using the hostname in putty and winSCP, but today, that hostname is unrecognized; I have to use the local IP. I checked the pi and the hostname is still valid on the machine, just not mapped on my network. Any idea what happened?

2) Also, I'm using a custom port, not 80, and that is working fine, but now when I go to my default DNS url. it opens the admin page of my router! I don't have http or 80 forwarded, so somehow this is a default. Does that make sense?

bren1818 commented 6 years ago

@merrickw fret not. Likely the Pi's ip address changed. I suggest on your router, lock it to an ip by MAC address. Then program the hostname and the IP in your host files if able.

This is also likely why you're seeing your router admin page, the routing to the pi is not happening. You're likely only seeing the router admin page because you're on your internal Network. As a precaution, open up a proxy site and try to reach your pi. It should not show your router page. If it does you should certainly configure your router to not allow that.

I'll be home later this tonight and follow up.

On Dec 5, 2017 11:17 AM, "merrickw" notifications@github.com wrote:

@sktaylortrash https://github.com/sktaylortrash , I did have some feedback on your original document if it helps ...

@bren1818 https://github.com/bren1818 , I tested the scene change and it does work, thanks for the fix.

Now for a couple of questions...

1.

Yesterday when I changed the hostname of the pi I had no problem using the hostname in putty and winSCP, but today, that hostname is unrecognized; I have to use the local IP. I check the pi and the hostname is still valid on the machine, just not mapped on my network. Any idea what happened? 2.

Also, I'm using a custom port, not 80, and that is working fine, but now when I go to my default DNS url. it opens the admin page of my router! I don't have http or 80 forwarded, so somehow this is a default. Does that make sense?

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/bren1818/TCPLightingWebInterface/issues/6#issuecomment-349355217, or mute the thread https://github.com/notifications/unsubscribe-auth/ACDCH6YgIL2CR7nsQI7XpFYr4nAOFRACks5s9WyCgaJpZM4LEhw5 .

sktaylortrash commented 6 years ago

@merrickw

for the networking stuff

merrickw commented 6 years ago

@bren1818 Actually, I locked down the local IP, so that didn't change. Just the hostname became invisible on my network ... maybe related to some local dns/hosts setting?

And I have no problem getting to the lighting page using the custom port; My problem was when I tested my URL without the custom port, (going to 80). But yes, the proxy test was a good idea, the admin page only comes up locally - thanks for that suggestion!

sktaylortrash commented 6 years ago

and you both replied before I did.

if you are using your router as the DNS server for your network. And have statically assigned the IP check if your router has a local hosts option for the DNS. Then you can set the Pi's hostname and IP and the router's DNS should resolve it

AinV07 commented 6 years ago

Wow, lots of activity here all of a sudden, the Black Friday effect continues! I too picked up some Google Home Minis and a SmartThings Hub 2. Slightly different challenge on my part, I wanted to integrate both the TCP lights and my legacy X10 stuff.

Since I already use Melloware's X-10 Commander app and agent, that meant I had local web server interfaces for both TCP (this project) and X10 (using X10 Commander's agent), giving me URL command capabilities for turning on and off devices (brightness control will be next weekend's project).

On the forums for SmartThings and X-10 Commander, I saw mention of another project called TriggerCMD... it's a SmartThings custom app that is cloud-based and has a local agent (I'm using the Windows agent, also available in Linux etc. so a Raspberry Pi implementation is the next step once I've got everything tuned the way I want). If you haven't checked it out, it's worth the time.

With TriggerCMD custom app installed on my SmartThings hub, my SmartThings hub integrated with Google Home, some simple batch files using cURL.exe to trigger URL-based commands, and TriggerCMD commands configured in the TriggerCMD agent to call those batch files, I was gleefully telling the Google Assistant to turn on/off all my TCP and X-10 stuff within a total span of about an hour last night. The plus side of all this... it works through any of my Google Assistants now, including my Android phone!

Off to work now, will try to curate a step-by-step guide through all of the various resources online I used to learn how to do it all.

sktaylortrash commented 6 years ago

@AinV07 that's awesome can't wait to see what you put together.

bren1818 commented 6 years ago

@AinV07 - thanks for sharing. Keep us posted. I listed a number of the commands here: https://github.com/bren1818/TCPLightingWebInterface/wiki/Bridge-API-Commands---For-Developer-info. Really curl is the key and knowing what the bridge commands are, and the IDs of the devices help. I'll be rolling out an update hopefully tonight or shortly there after which will have an "IFTTT" builder to assist with making the IFTTT applets. I dont have a smart-things hub, but there have been a few users on here who have asked about supporting it. I'm sure they'll be eager to read what you come up with. 👍

bren1818 commented 6 years ago

@sktaylortrash - One thing I thought to add to your instruction set for the pi, In the apache2.conf (/etc/apache2/apache2.conf) the following two lines should be added to the end of the file: ServerSignature Off ServerTokens Prod

And then Apache restarted (sudo service apache2 restart) This simply removes what kind of host is running behind the scenes. Its a small security measure, but a measure none the less.

sktaylortrash commented 6 years ago

@bren1818 I can definitely do that, not a directive I was a aware of

bren1818 commented 6 years ago

@sktaylortrash, @merrickw I pushed a new build out. It should look much more presentable now. I still have some things on my would like to do list, but I got excited... and tired, so I pushed what I have done so far. Hope you like it. If you dont, well, let me know either way : )

I will do some more extensive testing with the scheduler this week and write up docs on how to setup a crontab in Linux or use the Scheduler on Windows.

AinV07 commented 6 years ago

Draft of writeup for integrating X10, TCP, SmartThings, Google Assistant:

X10_TCP_ST_GA_writeup.txt

merrickw commented 6 years ago

@bren1818 , the UI changes look great.

I was actually tinkering with a page that had all rooms/devices/scenes on a single page; I can stop that tinkering now.

The IFTTT query builder is helpful too; And the handy project/wiki links as well. The new config file makes sense as well. Nice job!

I wonder if the TCP support folks are aware that you're keeping their product alive...

merrickw commented 6 years ago

@bren1818 I just noticed a difference from XAMPP and Apache; I tested on my laptop with XAMPP and saw all the style elements - black border, colors for the rooms; but when I migrate the same files to the RaspberryPi with Apache, I get a bare bones web page with no colors or styles. Any idea what the difference is?

sktaylortrash commented 6 years ago

@bren1818 looks great and the IFTTT builder will be super useful

@merrickw that'll be something either to do with file permissions or an apache module as it sounds like CSS isn't rendering or something. There must be a significant difference in the gui raspbian vs the lite, because I'm just not seeing that on my pi install. Well that or the version of NOOBS canakit is shipping is super old and different. I downloaded it fresh. Again hopefully my zero arrives tomorrow as scheduled so I can go through everything. I ordered the same kit as you guys so hopefully it's the same experience

@AinV07 at first glance that's a pretty comprehensive document. I'll see if I can take a swing at it this weekend.

merrickw commented 6 years ago

@sktaylortrash I just opened the page from my desktop and I the style shows properly. Maybe it's a browser issue on my laptop; not sure, but not important either!

bren1818 commented 6 years ago

@merrickw - It is/was very likely a cache issue. In the previous build I had been using a CDN to pull some of the javascript libraries. In the current build I downloaded, minified and merged those files since the majority of users will be running this locally. This also serves those users who may block outbound requests for their IOT projects. In future if you run into issues like that, try pressing ctrl + F5 to force refresh the page.

merrickw commented 6 years ago

@bren1818 Bingo. Ctrl-F5 resolved the issue. Thanks!

vijimaini commented 6 years ago

@bren1818 I finally used letsencrypt to get TLS signed certificates which I installed on Apache. Good learning experience. Problem I had was I got invalid port when I used it from IFTTT. I found that include.php has only provision for 1 external port, which I was using for http. Can I use multiple ports? if yes how. I tried Redirect in httpd-vhosts.conf to redirect http to https. It works internally but externally gives same error "Invalid Port". BTW: I have ordered Raspberry Pi Zero Wireless. It should come in next week, so will use the Guide. Thanks @sktaylortrash for the Wiki Guide. Looks Great!

bren1818 commented 6 years ago

@vijimaini If you're using https, its very likely ifttt will either expect no port to be specified (if https is in your address), or port 443 versus the non-standard one with https. With Apache & vhost configuration, (I'll have to double-check for the pi) there should be a second config file 'ssl.conf' which points to your vhost directory and shows it uses 443.

in your default vhost file you could specify more than one host ie: :80, :1818 but you'd have to tell the httpd.conf to listen on the non standard ports (ports other than 443, or 80).

Hope that helps!

vijimaini commented 6 years ago

@bren1818 Thanks for above info. On IFTTT I am using port 10000 for http and 10010 for https. On My firewall I port forward as 80 and 443 to my apache server. I am also using security in file include.php to restrict external ports that you designed. However it only allow for either 1 port when using "RESTRICT_EXTERNAL_PORT", is set to 1, or value 0 allows all ports. My question was, can we have say few ports accessible when using "RESTRICT_EXTERNAL_PORT", set to 1. Another question: Is there a api to set date/time automatically using cron job. I saw there is a manual set date time feature.

Nice to see lot of action!!

sktaylortrash commented 6 years ago

For those looking to secure external access to this and other IOT devices a bit more. This reverse proxy setup might be of interest. http://tinkerman.cat/secure-remote-access-to-your-iot-devices/

If you're unfamiliar with the concept of reverse proxy basically this box would be the one you open ports on your firewall to. Then you would setup paths from that box to your internal systems. The way the tutorial sets it up you would create a virtual path on the server that redirects to an internal host. So say you setup https://myhome.no-ip.com as your external url you could define that https://myhome.no-ip.com/tcp transparently passes to http://lighting.local internally

If you're only intending to run this project it's not a big deal but if you are like me I've assembled and bought some web accessible relays to perform other tasks like button pushes and device restarts. These devices typically have only enough processing power for basic http interfaces so enabling https on them would cause delays and halts.

So with the proxy I could setup an IFTTT command that connects to https://myhome.no-ip.com/relay1 and issue get commands of say ?on ?off or ?toggle

sktaylortrash commented 6 years ago

@bren1818 is it possible to have a flag in config.inc.php that determines whether or not the IFTTT builder should generate http or https urls? It's not a huge thing just took me a beat to figure why my new commands weren't working

vijimaini commented 6 years ago

@sktaylortrash Thanks. I will look into using reverse proxy for my project.

bren1818 commented 6 years ago

@sktaylortrash the IFTTT builder currently solely uses the EXTERNAL_DDNS_URL and EXTERNAL_PORT to make the requests, well in conjunction with the various flags. I suppose I could put a toggle in for whether to use the local / external address.

@vijimaini - the restriction just forces the external port to be compliant. The rest relies on the .htaccess file. If you need another port, edit the api.php line 22, and make it:

if( ($_SERVER['SERVER_PORT'] != EXTERNAL_PORT) && ($_SERVER['SERVER_PORT'] != "otherport") ){

Where otherport would be the other port you want.

Re: the setting the date and time, that function is to correct the TCP Bridges date and time. It sometimes gets out of sync. Your computer should have a valid date and time, however in the config.inc.php make sure you update the date_default_timezone_set to match your timezone as well as your Latitude and Longitude. A list of timezones can be found here: http://php.net/manual/en/timezones.america.php

sktaylortrash commented 6 years ago

@bren1818 my bad I set https on internal but not external that sorted it

vijimaini commented 6 years ago

@bren1818 Thanks. I will make changes in my api.php