Fauxmo as a binary ?

[mocallins]

Hi Nate.

Any chance you could compile Fauxmo ? I think you can do it with Python interpreter.

Will make it so much easier to use, not having to install python3 blah, blah, blah.

I think for dev work, definitely still install Python.

Any other thoughts on making this Fauxmo completely yours and changing from using .json and just make it a python dictionary ?

[n8henrie]

Hi!

Any chance you could compile Fauxmo ? I think you can do it with Python interpreter.

No, Python is not a compiled language. You can "compile" it to bytecode (.pyc files), but these still need to be run by a python interpreter. However, fauxmo has no external dependencies, which greatly simplifies things, and many (most?) operating systems either come with python3 preinstalled or make it incredibly easy to install. What OS are you on that you're having difficulty installing a python3 interpreter?

I have considered rewriting the project in a compiled language (RIIR syndrome), but it would be extremely low priority (since things work fine as is).

One could use something like pyoxidizer or nuitka to compile a single binary that has both the python runtime and Fauxmo, but honestly even this seems like a silly amount of work for a project that is so easy to install.

To me, it seems like the comparison is:

• On one hand, install python3 (probably as simple as sudo apt install python3.9-full), then python3 -m pip install fauxmo
• On the other hand, scroll through the Fauxmo releases to find the version that matches your architecture, download with wget (or through your browser and then rsync it over, either of which could need to be installed)

To me, it doesn't seem like much saved effort!

Any other thoughts on making this Fauxmo completely yours

It's already completely mine, I've rewritten nearly the whole codebase 😉. And yours, since it's MIT licensed.

changing from using .json

I chose to use json because it's well understood, easy to read, and has tooling in the Python stdlib.

and just make it a python dictionary

I'm not sure what you mean. Like just change the .json to .py and then import the config as a separate module? Json translates readily into a Python dictionary, which is one of the advantages of using json.

[mocallins]

Hmmm well i couldn't sleep and got bored, and looked around at some of the compiling options.

I'm on a Raspberry Pi Zero W.

I end up using pyInstaller.

pip install PyInstaller

needed some extra libs,

apt-get install python3-dev python-dev upx

got it to compile with

pyinstaller fauxmo -F --collect-submodules fauxmo.plugins --onefile

Works on the system i built it on, and will just copy the compiled version to 1 or more of my other machines, and see if all is good.

I was thinking is you switched to a python dictionary (very similar syntax to json), natively, then you could embed functions calls or variables right in the dictionary that would get interpreted correctly, and substituted properly. Could also the use the r"String" , and really put anything you want in it and then massage it with any functions.

Just some thoughts for possible improvements, to make it more robust.

[n8henrie]

Glad to hear that pyinstaller works! Thanks for providing the command you used, I'm sure others will find it helpful. I've had some problems with PyInstaller on MacOS before -- issues with the framework builds. When I need to make a Python binary lately I've turned to pyoxidizer, though I've heard a lot of good about nuitka (both linked above).

I assume that if you want to add a new plugin you have to recompile the binary?

I was thinking is you switched to a python dictionary

I guess I'm still not sure what you mean by this. How and where would the dictionary be stored? pickle? The issue is serialization to disk, and how end users would edit the config file. Did you see my question above about storing as a python module? Is that what you mean? If not, can you give me a more concrete example?

Could also the use the r"String" , and really put anything you want in it and then massage it with any functions.

Can you be more concrete about what you're missing? I remember your concerns about windows path quoting from #109, it seemed like you were struggling to understand escaping backslashes. Did you end up making a helper module like we discussed?

[mocallins]

I assume that if you want to add a new plugin you have to recompile the binary?

Yea I'm sure. And if you end up making any other change, yes recompiled, but once i got everything together, no problem.

[mocallins]

I guess I'm still not sure what you mean by this. How and where would the dictionary be stored? pickle? The issue is serialization to disk, and how end users would edit the config file. Did you see my question above about storing as a python module? Is that what you mean? If not, can you give me a more concrete example?

I was thinking in some of my python work, we had a need to have a python dictionery in a file, and i can't remember what we used to read it back in. Maybe something as stupid as line by line , then did an eval, maybe. smh, that was like 3 jobs ago, lol, just can't remember the details. But looking at the .json file make me rethink it. pickle, sounds familiar too, i'm just not sure. I just know i had a need for it, so i worked on it, just don't remember how it got completed.

I mean once you stringify it, you can split it up any way you like, and that seems to ring a bell. And i'll bet it was before i found (prettyPrint ?, maybe)

I'm not exactly sure what you mean by serialization ?

[mocallins]

Can you be more concrete about what you're missing? I remember your concerns about windows path quoting from #109, it seemed like you were struggling to understand escaping backslashes. Did you end up making a helper module like we discussed?

No i did not. Yes the backslashing and quoting was a pain. And then you have to make sure its actually finding the command correctly, then fauxmo works. Up til then, just beating your head.

I'm much better at delpoying it now, on my own Raspberry Pi's, lol

That's why the whole setup of Python3 and the venv, blah blah blah. Just 1 executable file now

[mocallins]

I guess I'm still not sure what you mean by this. How and where would the dictionary be stored? pickle? The issue is serialization to disk, and how end users would edit the config file. Did you see my question above about storing as a python module? Is that what you mean? If not, can you give me a more concrete example?

I was thinking in some of my python work, we had a need to have a python dictionery in a file, and i can't remember what we used to read it back in. Maybe something as stupid as line by line , then did an eval, maybe. smh, that was like 3 jobs ago, lol, just can't remember the details. But looking at the .json file make me rethink it. pickle, sounds familiar too, i'm just not sure. I just know i had a need for it, so i worked on it, just don't remember how it got completed.

I mean once you stringify it, you can split it up any way you like, and that seems to ring a bell. And i'll bet it was before i found (prettyPrint ?, maybe)

I'm not exactly sure what you mean by serialization ?

Also you would edit the file, just as you do the .json file. Gah i wish i could remember how we read that back in.

[mocallins]

I also did copy it over to a couple of other machines (rPi) i have, and is working, no problem.

[n8henrie]

line by line , then did an eval... I mean once you stringify it, you can split it up any way you like, and that seems to ring a bell.

I have to be frank, this sounds... awful! 😆 I don't say that to be unkind, but it sounds very hackish, and potentially unsafe (eval -> security issue). I would want to stay away from that with a 10 foot pole.

I'm not exactly sure what you mean by serialization ?

This might be a good topic for a little reading! Serialization is a very interesting topic, in simple terms it's how one approaches taking a data structure in memory (such as a dictionary in python) and turning it into a storage format on disk (a text file with json content in this case). There are a lot of different approaches, each with advantages and drawbacks. I recommend you do some reading on the pickle module -- it's very handy! But it can post some security issues if you are unpickling untrusted files -- it can potentially run arbitrary code.

Yes the backslashing and quoting was a pain. And then you have to make sure its actually finding the command correctly, then fauxmo works. Up til then, just beating your head.

It's possible that Fauxmo is not a great fit for your case. It's mostly intended for an audience with some level of comfort with Linux, python, and the command line. Being able to find the path to tools and knowing how to properly escape json is kind of table stakes for a project like this. And it sounds like you've been able to sort this all out, which is great! But it's not really a project that is geared toward people that aren't comfortable with these ideas.

Also, if you're struggling, don't forget that little script I wrote you in the other thread to make it easier to convert paths from a Python dictionary to json format.

That's why the whole setup of Python3 and the venv, blah blah blah.

Again, this is kind of table stakes for working with a python project. I'm sorry if you think it's too much effort, but I respectfully disagree; I think the steps in the readme have been laid out in a straightforward way.

Just a thought, perhaps I could enable a Wiki for fauxmo? Would you be interested in contributing the things you've learned along the way? I think that might be a good approach to helping others that are less comfortable with Python. What do you think?

I also did copy it over to a couple of other machines (rPi) i have, and is working, no problem.

Great, glad to hear it!

[mocallins]

Yea, i don't mind contributing to a Wiki. I've got little to nothing else to do all day, lol I would bet, if you got down into the nuts and bolts of the json package, they do exactly line-by-line.because i can't really imagine any other way to deal with a text file. And no disrespect. It might sound hackish, but in reality as a formerly paid professional test engineer, finding a solution that works is paramount.And as you know you forked this project because you just had a better idea, that's all i'm suggesting as well. Computers and using software, even for engineers, does not have to be the most difficult thing on the planet, imho.

On Wednesday, October 27, 2021, 12:41:17 PM CDT, Nathan Henrie ***@***.***> wrote:  

line by line , then did an eval... I mean once you stringify it, you can split it up any way you like, and that seems to ring a bell.

I have to be frank, this sounds... awful! 😆 I don't say that to be unkind, but it sounds very hackish, and potentially unsafe (eval -> security issue). I would want to stay away from that with a 10 foot pole.

I'm not exactly sure what you mean by serialization ?

This might be a good topic for a little reading! Serialization is a very interesting topic, in simple terms it's how one approaches taking a data structure in memory (such as a dictionary in python) and turning it into a storage format on disk (a text file with json content in this case). There are a lot of different approaches, each with advantages and drawbacks. I recommend you do some reading on the pickle module -- it's very handy! But it can post some security issues if you are unpickling untrusted files -- it can potentially run arbitrary code.

Yes the backslashing and quoting was a pain. And then you have to make sure its actually finding the command correctly, then fauxmo works. Up til then, just beating your head.

It's possible that Fauxmo is not a great fit for your case. It's mostly intended for an audience with some level of comfort with Linux, python, and the command line. Being able to find the path to tools and knowing how to properly escape json is kind of table stakes for a project like this. And it sounds like you've been able to sort this all out, which is great! But it's not really a project that is geared toward people that aren't comfortable with these ideas.

Also, if you're struggling, don't forget that little script I wrote you in the other thread to make it easier to convert paths from a Python dictionary to json format.

That's why the whole setup of Python3 and the venv, blah blah blah.

Again, this is kind of table stakes for working with a python project. I'm sorry if you think it's too much effort, but I respectfully disagree; I think the steps in the readme have been laid out in a straightforward way.

Just a thought, perhaps I could enable a Wiki for fauxmo? Would you be interested in contributing the things you've learned along the way? I think that might be a good approach to helping others that are less comfortable with Python. What do you think?

I also did copy it over to a couple of other machines (rPi) i have, and is working, no problem.

Great, glad to hear it!

— You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub, or unsubscribe. Triage notifications on the go with GitHub Mobile for iOS or Android.

[n8henrie]

finding a solution that works is paramount

Agreed! While json does have some issues, it works well, which is why I went with it.

I would bet, if you got down into the nuts and bolts of the json package, they do exactly line-by-line.because i can't really imagine any other way to deal with a text file.

Here's the source. Take a look! https://github.com/python/cpython/blob/bb3e0c240bc60fe08d332ff5955d54197f79751c/Lib/json/__init__.py#L274

Yea, i don't mind contributing to a Wiki.

Excellent! https://github.com/n8henrie/fauxmo.wiki.git

Another thought I had, if you wanted to contribute the ability to use configparser as an alternative config format, I'd consider it given that it's also in the stdlib. I haven't looked into the feasibility, but it should be more human-readable and I think should give you fewer headaches about backslash escaping. In order to merge a PR for this I would definitely want tests showing that it works and that it doesn't affect the current json config approach. Let me know if you're interested.

[mocallins]

Hmmm me personally, i never really had much luck with configparser. It always seemed a little restrictive, if i remember right. But it is absolutely a possible approach.

So it finally dawned on me what you meant by using a "module" for the config file. That would be an excellent approach, since it would all be native python at that point.

Here's what the change would look like in my Windows case:

import os

config = {
  "FAUXMO": {
  },
  "PLUGINS": {
    "CommandLinePlugin": {
      "DEVICES": [
        {
            "name": "monitor",
            "port": 49920,
            "on_cmd": "echo monitor on",
            "off_cmd": os.path.realpath(r"c:\NirSoft\nircmdc.exe monitor off"),
            "use_fake_state": true
        },
        {
            "name": "youtube",
            "port": 49921,
            "on_cmd": os.path.realpath(r"C:\Program Files\Mozilla Firefox\firefox.exe youtube.com",
            "off_cmd": "echo youtube off",
            "use_fake_state": true
        }
      ]
    }
  }
}

So the only things different, as you can see, used an import, and the definition of the config dictionary (i just used "config =", You definitely could name it any variable name, which would be an absolute requirement, just as the dictionary elements "FAUXMO" and "PLUGINS", blah blah blah.

This really opens a security question, because you would be importing code, for real. the import could absolutely be done away with then the *_cmd could just be a string or r"", if wanted, but easily just a quoted string.

Now this brings up the issue of dynamically importing a module, that is not in the PYTHONPATH, which i have never done before. Everything i worked, we put things in static places to be part of packages, blah.

So here's a reference i found, easy read, you tell me what you think, lol. https://levelup.gitconnected.com/import-your-own-python-code-without-pythonpath-tricks-9068495c1bba

it would be part of the installation requirement, i think, and still would have a structure, you'll see when you read. But a pretty simple definition and would restrict the "-c" option a little, or a lot, lol.

And the way i read it, you would do it once at installation, then you just keep putting your "configuration module" in the same place.

[mocallins]

Hmmm me personally, i never really had much luck with configparser. It always seemed a little restrictive, if i remember right. But it is absolutely a possible approach.

So it finally dawned on me what you meant by using a "module" for the config file. That would be an excellent approach, since it would all be native python at that point.

Here's what the change would look like in my Windows case:

import os

config = {
  "FAUXMO": {
  },
  "PLUGINS": {
    "CommandLinePlugin": {
      "DEVICES": [
        {
            "name": "monitor",
            "port": 49920,
            "on_cmd": "echo monitor on",
            "off_cmd": os.path.realpath(r"c:\NirSoft\nircmdc.exe monitor off"),
            "use_fake_state": true
        },
        {
            "name": "youtube",
            "port": 49921,
            "on_cmd": os.path.realpath(r"C:\Program Files\Mozilla Firefox\firefox.exe youtube.com",
            "off_cmd": "echo youtube off",
            "use_fake_state": true
        }
      ]
    }
  }
}

So the only things different, as you can see, used an import, and the definition of the config dictionary (i just used "config =", You definitely could name it any variable name, which would be an absolute requirement, just as the dictionary elements "FAUXMO" and "PLUGINS", blah blah blah.

This really opens a security question, because you would be importing code, for real. the import could absolutely be done away with then the *_cmd could just be a string or r"", if wanted, but easily just a quoted string.

Now this brings up the issue of dynamically importing a module, that is not in the PYTHONPATH, which i have never done before. Everything i worked, we put things in static places to be part of packages, blah.

So here's a reference i found, easy read, you tell me what you think, lol. https://levelup.gitconnected.com/import-your-own-python-code-without-pythonpath-tricks-9068495c1bba

it would be part of the installation requirement, i think, and still would have a structure, you'll see when you read. But a pretty simple definition and would restrict the "-c" option a little, or a lot, lol.

And the way i read it, you would do it once at installation, then you just keep putting your "configuration module" in the same place.

Or just keep updating the one that's there.

[mocallins]

So i didn't think to check that code directly in the interpreter. I missed a closing ")" on the second off_cmd. And the word "true" needs to be "True", or at the top you could do "true=True" those fixes and it stored perfectly in the interpreter.

lol, wow this is making me work, thinking Python again ,lol

[mocallins]

Hmm just in this timeframe of discussing this, i don't know if i like the python module approach. Just not sure of the security issues. I mean i do realize they would only be doing it to themselves, but would it give a superior hacker the ability to damage there Alexa device, or affect AWS somehow ? Yea, hmmmm, a few unknowns there, smh.

[n8henrie]

So it finally dawned on me what you meant by using a "module" for the config file.

Excellent. Yes, this is the approach Flask used to take (maybe still takes?); it can be a reasonable approach for a developer project where one is already writing Python code.

i don't know if i like the python module approach.

Yes, generally I don't care for the python module approach, I will probably not accept a PR to implement this. To me, it is easily as complex as the current json approach (even if it seems easier in your case), and configuration should not be executable code IMO. Obvious as MIT code you're more than welcome to fork and take this approach if you'd like!

I still think configparser might be feasible, but it would be a nontrivial amount of work between the implementation and most especially tests to help ensure correctness.