Maximum SELF_DESTRUCT_AGE limit

[RussW]

@sozzled define ( '_FPA_SELF_DESTRUCT_AGE', 1 ); // age of FPA file before _FPA_SELF_DESTRUCT runs (set as CONSTANT so it can't be changed/overridden at runtime)

One day should be enough. If the file is auto deleted, the owner can put a new version there. Reducing the time wait reduces the opportunity for someone else running the FPA and (possibly) obtaining some private information.

[RussW]

The _FPA_SELF_DESTRUCT_AGE of 7 days was an arbitrary choice I made at the time, I guess it's up for discussion, but I wouldn't want it lower than 4-5 days, otherwise users may end up having to keep uploading fpa if troubleshooting takes a little longer than usual and I think it will bug the crap out of them and they'll stop using it.

[sozzled]

The self destruct mechanism doesn't actually do anything except define a constant. The constant is referenced later (in two places) but the definition of the constant, by itself, is not a trigger to delete the file.

[RussW]

if the _FPA_SELF_DESTRUCT constant is defined (uncommented) then fpa checks itself for the last modified date, if that date is over the _FPA_SELF_DESTRUCT_AGE setting, then _FPA_SELF_DESTRUCT_DOIT is defined, which then tells the delete script to run

[sozzled]

Can you please show me? Do you use Skype? I checked the logic. I tried it in practice. It doesn't work.

[sozzled]

Change these 101-102 as a test:

define ( '_FPA_SELF_DESTRUCT_AGE', -7 );       // age of FPA file before _FPA_SELF_DESTRUCT runs (set as CONSTANT so it can't be changed/overridden at runtime)
if ( defined('_FPA_SELF_DESTRUCT')  ) {

run it like that

[RussW]

Probably a very dumb question, but... Quite why would I want to change that constant to "-7". when the routine is setup to look for a positive difference integer and check for an "inverse" setting in the datetime array!

I think you need to find a Windows utility/library (on unix systems, we use "touch" to manually modify create, access & modify dates) that allows you to change the files "modified date" so you can copy fpa to a folder, then without having edited it, change the modified time to older than _FPA_SELF_DESTRUCT_AGE and then run fpa.

I've tested this on Apple OSX, CentOS, CloudLinux and Tails (hardened) Linux distro's (CloudLinux with suhosin) and commercial unices - Oracle Solaris, HP-Ux, IBM Aix, IBM Z with a Redhat Linux container and it has worked just fine for me so far.

so I can only suggest that my limited knowledge of the Windows environment is causing the problem here.

[sozzled]

Russ: call me, please. Do you want my Skype address, telephone number? We could solve this (or be on the way to solving this) in a few minutes. Several other discussion points you've raised as gitissues could also be addressed in short order with an online group chat. Besides, I'm a bit tired of playing solitaire on the computer while I'm waiting for a reply.

[sozzled]

OK. I found a utility to change the modified date of a Windows file. I changed the modified date (to a week ago), ran the FPA and it automatically deleted the file when I tried to run it. Thank goodness.

Therefore, I can confirm that the work-in-progress version auto self-destructs as committed. Thanks.

I still think 7 days is too long, especially when there's a big red warning that, effectively, says, "prolonged exposure of this script in your filesystem could result in allowing other people to snoop around" (and that's why I think one day is ample time).

[RussW]

(phew)... TFFT... I was beginning to think I was going nuts here and every development system I have access to was doing something funky!

This is never going to be a perfect solution and always a "happy but uneasy compromise", so I guess we're up for a vote folks, highest wins: [EDIT] "uneasy medium" changed to "uneasy compromise"

_FPA_SELF_DESTRUCTAGE definition? (auto-delete fpa when "x" days old)_

• 5 days old 👍 RussW, 👍 @mandville (don't want to piss-off the users)
• 4 days old
• 3 days old (or absolute worst case RussW)
• 2 days old
• 1 day old 👍 @sozzled (as ½ day isn't an option)

[sozzled]

Here's the thing: it's 6:50 pm and I've been working (or rather waiting) on the FPA since I woke up at 2:00 am this morning. Forgive me if I'm a little ratty.

There's an official looking warning (there's always been a warning) about DELETING THE FPA after you use it. Muggins me, does that, just fine thank you. I don't keep that kind of spooky stuff around on any of my websites.

Other people ... well ... they do whatever they want to do and, nine times out of ten, they use the FPA once and that's it; it's all over, Rover.

For a very select few, they want to be comforted in the knowledge that they can run the FPA continually (every few hours or days) for the next while without having to spend 20 seconds to copy a file from somewhere.

Forgive me for not reaching for a box of Kleenex to wipe the tears of folk who would be inconvenienced by spending such an inordinate amount of time replacing a throwaway file.

This kind of gets silly, doesn't it, anticipating the likelihood that people may get antsy if the FPA script is automatically deleted before they've finished with it. How long would the average punter require to keep using the FPA (without spending that 20 seconds copying a saved version into their filesystem if it were automatically removed because they were ... well ... not finished using it)? An hour? A few hours? A day? A working week? More?

My bet is that people use this thing once. That's it. They don't use it again or, if they do, they use it in a month or two's time. So, my money is on a max of 1 day.

Me? Whenever I run the FPA it's because there's a new version that needs to be tested. I download it, use it, post the BBcode on the forum, then I trash it.

You could put an option into the FPA script to ask a question (How long do you want to keep me?) and store the result as a cookie. You could put leather trim on the dashboard and have double overhead foxtails on the radio antenna ...

I hear a Monty Python farce playing in the next room.

[RussW]

TL;DR asked for a simply, effective, efficient vote, not a "diatribe". Assumed voted 1 day

[sozzled]

Could have been resolved in one minute with a teleconference hookup. Asking for a vote makes this look serious.

My vote: half-a-day.

[mandville]

[quote]How long do you want to keep me?) and store the result as a cookie[/quote] that opens cookie laws. 7 days would be good enough for most forum work.

[RussW]

assuming next closest option for @mandville 5 days if thats ok with you (I tried to "hit a happy medium" with the highest being 5 days, between my initial choice of 7 days and @sozzled initial suggestion of 1 day)

[sozzled]

No further comment from me. Suggest re-wording the official-looking wording that says

"Due to the highly sensitive nature of the information displayed by the FPA script, it should be removed after use. If the script is left on your site, it can be used to gather enough information to hack your site."

to

"The FPA script may contain private information that could be used to obtain information by others to compromise your website. We recommend that you remove the FPA script after you use it."

That should should be enough. :smile:

[mandville]

agree on the re wording - "The FPA script may contain private information that could be used to obtain information by others to compromise your website. We recommend that you remove the FPA script after you use it." 5 days self destruct works for me ( thats calendar and not business days) the script is free so they can just upload it again

[sozzled]

Could be a problem here with reusing the downloaded copy of the FPA.

1) Download the FPA (zip package when available), extract the file fpa_en.php (store the extracted file on your PC) and upload the extracted to the server. 2) What is the "last modified" date of the extracted (locally stored) file? 3) If the last modified date of the extracted file is "old", is this date carried over to the server? (The date carries over in a Windows environment but if you use FTP it seems to be OK, I think.) 4) Whenever someone tries to run the FPA on the server (and the auto self-destruct check detects that it's older than n days) the file fpa-en.php will be deleted and the FPA can't be used. 5) Uploading the same extracted file from the local machine to the server results in the same thing (at least if copying files from one place to another in a Windows environment). 6) Will extracting the file from the zip package again apply a new "last-modified" date (I think it will, but I haven't tested it). Does this mean that people will have to extract the file from the zip before they have a copy that won't self-destruct when it's used?

I don't know. While we're in a testing phase and we're testing new versions of fpa-en.php that are changing every day or so, this isn't an issue. I encountered the problem when I copied a version of fpa-en.php that I had created from the last commit, saved it to my local machine (changed the last-modified date to something old), copied this file to my Wampserver and it automatically deleted itself.

Seems to be OK if you take the local copy, upload to a real website using FTP, and today's date is the "last modified" date. No good, however, if you take the local copy, store it on the remote host (but not in the place where it will be used) and then copy the file from one folder on the remote host to the test environment; the file will be deleted if you try to run it. Requires having to upload the file from a local store to the remote store when you need it.

[RussW]

Well caught @sozzled As you say, unzipping of the file itself may not be enough to change the "modified" date, nor simply uploading the same copy again to the server.

I'll have a tinker later today and see what I can determine.

[mandville]

uploading a file resets the date on the server, else you would have overwritten joomla files from 2015 etc.

[sozzled]

Thanks. "Auto self destruct" in a local environment is a show-stopper. Seems to be OK if FTP-ing to a remote server.

I am available anytime to talk-walk through issues on Skype (my Skype username is [removed]).

[RussW]

yes, because FTP and the server has no previous metadata on the file, the "modify time" is reset to todays date and time, but... on your own local machines, if files are copied to another location, the current metadata is maintained by the operating system, and the modified date and time is the same.

I guess, in reality, autodestruct on a localhost isn't actually needed, is it? So if we test for IP Address "127.0.0.1" or "::1" or (not and, as may have changed hostname)_ hostname of "localhost" the self_destruct is not invoked.

[sozzled]

I don't know, Russ. I would have to try this out.

FYI, I run Wampserver on another [Windows] PC in my LAN. I view/transfer files from the PC I'm writing to you from to the other PC through a "share". I then access the "remote" host in the LAN (via the browser) by using an alias host name (defined in my hosts file) but not as 127.0.0.1 (because the Wampserver "server" is located on a different IP address in my LAN).

So I'm not sure if a 127.0.0.1 check would be sufficient.

[frostmakk]

And you could be using different ports. On my Wamp I have mariadb on localhost:3307

[sozzled]

As I wrote before, I can walk-talk through the issues on Skype.

I'm not changing the IP ports. This could be addressed if you knew the local subnet (e.g. 192.x.x.x or 10.x.x.x) for the LAN, compare the LAN subnet used on the device where the FPA is run with the subnet of the target and then do something). Beyond my capabilities at this time to know how to access the local subnet information in PHP; I would need to research it.

[RussW]

So am I correct in saying that your home network is using either a subnet range of 192.168.x.y or 172.16.x.y or 10.x.y.z ? and your localhost web-server is either 127.0.0.1 or one of the previously mentioned address ranges?

These ranges are "reserved" for local networks, so will never be used online, which means we if any one of these are being used, it must be a home dev environment.

[sozzled]

Sort of correct, Russ.

The local IP addr of the PC I'm typing from is 10.1.1.39; the IP addr of the target [Wampserver] is 10.1.1.37. Obviously, my PC can also be accessed using ::1 (or 127.0.0.1).

Correct that these addresses are never used online. We're only talking about a home dev environment but that's what a lot of forum users like to use when they post the FPA report.

We need to know the local IP addr for the device that runs the FPA and the local IP address of the server; not the NAT addr.

Looks like this discussion on https://stackoverflow.com/questions/3219178/php-how-to-get-local-ip-of-system may have few clues.

[frostmakk]

I'm not changing the IP ports.

I have not suggested that you change ports sozzled. I have simply explained why searching for "localhost" or "127.0.0.1" without a wildcard at the end could fail.

[RussW]

I'm a little lost as to how we got to discussing ports, but this is what I've come up with to determine if the user is local host or not;

if ( $_SERVER['REMOTE_ADDR'] =='127.0.0.1' OR $_SERVER['REMOTE_ADDR'] == '::1' OR strpos($_SERVER['REMOTE_ADDR'], '192.168.', 0) OR strpos($_SERVER['REMOTE_ADDR'], '10.', 0) OR strpos($_SERVER['REMOTE_ADDR'], '172.16.', 0) OR $_SERVER['SERVER_NAME'] == 'localhost' ) {

That certainly covers the reserved subnets, make sense?

[sozzled]

Sorry, @frostmakk. You're right: I could be using different IP ports. I agree with @RussW that IP port usage on the target host is probably not relevant to the question.

I have a question: I've repeated my desire to talk through these things via Skype but it appears no-one is interested or no-one as, at least, said it's not going to happen for them.

[sozzled]

@RussW: I think we would be catering only for the selected reserved IP subnets (on target "hosts") but not addressing the question.

The question is, essentially, "I have a LAN in which I can access my website(s)—whether those websites are served on the same device as I'm using the browser or on a different device. How can I check if I'm working within the LAN for the purposes of knowing if running the FPA from any browser in that environment should check the last modified date or not to stop the file from being deleted when I try to use it?"

[mandville]

whenever i see people running localhost i cringe as its not a proper working environment especially home hosting. i dont see that the self destryuct should care about the IP its on. as for skype, my working and home life normally prevents me from spending that amount of time and i need to set up the dictaphone

[sozzled]

I agree with you, @mandville (@Webdongle would probably disagree with you) about what is, or is not, a "proper working environment". I have "drawn the crabs" when I wrote https://forum.joomla.org/viewtopic.php?t=925348

However, the self-destruct feature will bomb the FPA if it's used in a LAN-based (or "home dev.") environment. That's the point of this discussion.

[RussW]

@sozzled I'm not sure I fully understand your statement/question...

IF the webserver/website is on a local subnet, it will be using 127.0.0.1, ::1, 10.x.y.z, 192.168.x.y or 172.16.x.y as it's $_SERVER['REMOTE_ADDR'] thats the nature of internal lans.

If you access that webserver/website from inside your own lan (whether being hosted on the same computer as the browser you are using or a different computer on the same lan)_ then we'll plan on not running the _FPA_SELF_DESTRUCT because the $_SERVER['REMOTE_ADDR'] will be one of the reserved subnets.

IF that same setup is also being "home served" through your NAT/PROXY/ROUTER to the internet, and you access it through a browser to your now external IP or domain, it will then have that external IP Address in the $_SERVER['REMOTE_ADDR'] and the _FPA_SELFDESTRUCT will be enacted (which is probably a good thing as it's an external visitor)_

Does that make sense?

@RussW: I think we would be catering only for the selected reserved IP subnets (on target "hosts") but not addressing the question.

The question is, essentially, "I have a LAN in which I can access my website(s)—whether those websites are served on the same device as I'm using the browser or on a different device. How can I check if I'm working within the LAN for the purposes of knowing if running the FPA from any browser in that environment should check the last modified date or not to stop the file from being deleted when I try to use it?"

[sozzled]

Hold on ... I have an idea that might cut through the problem (I have to thank my wife—who knows little about webcraft—for acting as a sounding-board).

Instead of checking the date last modified and automatically deleting it, place a screen alert that says something to the effect, "This copy of the FPA was last modified more than x days ago and it will be deleted. Press OK to Delete it or Cancel to keep it."

The problem, however, is that anyone (other than the person who copied the file to the "server") who tries to run the FPA will be hit with that query and they'll just press "cancel" and then there's no real benefit.

There are other ways to deal with the issue, however, if there was the ability to "flip a switch" (would need some cookie store method to determine the current state of the switch) to disable the self-destruct. The self-destruct will not operate if using DEV or DIAG mode. I created a new wishlist item to turn on/off DIAG and DEV modes without having to modify the source.

[frostmakk]

I'm a little lost as to how we got to discussing ports, but this is what I've come up with to determine if the user is local host or not;

The port is part of the hostname. You will not catch this with if ( $_SERVER['REMOTE_ADDR'] =='127.0.0.1'.... Use strpos on '127.0.0.1' and 'localhost' as well.

[sozzled]

(no one is listening to me)

It won't work, @frostmakk.

What if the target (hosted) server is not "localhost" or "127.0.0.1" but on a different addr. in the LAN?

[sozzled]

I have another idea to allow (or disallow) auto self-destruct. Needs a cookie to make it work, however.

[sozzled]

IF that same setup is also being "home served" through your NAT/PROXY/ROUTER to the internet, and you access it through a browser to your now external IP or domain, it will then have that external IP Address in the $_SERVER['REMOTE_ADDR'] and the _FPA_SELFDESTRUCT will be enacted (which is probably a good thing as it's an external visitor)_

That may work. Would need to see it in action.

[RussW]

here's the new conditional statement, it's looking for the reserved subnets, internal IPV6 address (::1) and a substring of localhost in the address. If any of these are matched, we set $isLOCALHOST from "0" to "1" and the _FPA_SELF_DESTRUCT does not get invoked, else $isLOCALHOST = 0 and the routine looks for the modified date and runs as necessary.

strpos( $_SERVER['REMOTE_ADDR'], '127.0', 0 ) OR strpos( $_SERVER['REMOTE_ADDR'], '10.', 0 ) OR strpos( $_SERVER['REMOTE_ADDR'], '192.168.', 0 ) OR strpos( $_SERVER['REMOTE_ADDR'], '172.16.', 0 ) OR strpos( $_SERVER['SERVER_NAME'], 'localhost', 0 ) OR $_SERVER['REMOTE_ADDR'] == '::1'

[sozzled]

Here's a really simple solution:

The first time the FPA is used with a browser, check of a cookie exists.

If the cookie doesn't exists, ask the question: "Do you want to enable auto self-delete?" (Store the answer in a persistent cookie)

Then check the value in the cookie with the test for auto self-deletion and if both tests pass, delete the file.

For as long as the file is left on the server, there may be an issue if someone else runs the FPA (and that's why I counselled setting the maximum delay between creation and auto self-destruct to a short time). However, for as long as the file exists, the file will not be self destroyed unless the user says that they want to destroy it).

Alternative questions could be asked, for example, "Are you using a home dev. environment?", store the result in a persistent cookie (regardless of whether the answer is factually correct or not) and then test. It's really only a show-stopper if the FPA is run on a LAN and the last-modified date is outside the range. It won't be an issue if the last-modified date is within range (and that's unlikely to occur if the file is FTP-ed to a remote "properly hosted" server).

[RussW]

Correct me if I am wrong here, but wouldn't dropping persistent cookies open FPA up to a world of pain with the likes of GDPR (particularly in the EU)?

[sozzled]

Cookies have nothing to do with the GDPR. There's a separate set of "laws" relating to cookies that pre-date the GDPR. Let's face it, without cookies, the whole of J! would capsize!

Anyway, for anyone who's a nervous nelly about this, you could put a note in the documentation about the use of cookies and, if people complain, then point to the note in the doco. There is no requirement for people to give active consent to the use of cookies (unlike the active consent required in the GDPR to hold personal data). If you don't like cookies then you can disable them. If you disable cookies, you can't use a J! website (anywhere).

I take it that no-one wants to use real-time teleconferencing and, so, I'll drop the matter (and remove my Skype contact information from this discussion).

[RussW]

My apologies, although GDPR does mention cookies, the governing law is the EU ePrivacy Law, either way, the question stands.

Does the dropping of persistent cookies get FPA in to the realms of notifications and all the rest of it? I'd prefer not to go down that route if we can help it.

[edit] in addition, the latest commit implements the suggested "localhost" conditional testing, if you would like to see how it goes in your environment, that would be great. It seems to have worked ok in mine.

[mandville]

Receive users’ consent before you use any cookies except strictly necessary cookies.

[sozzled]

Do what you like. Cookies are irrelevant. (If you want to know why, ask yourself the question, "Whose website are we talking about and who would be 'concerned' that cookies are being used?") GitHub uses cookies (mostly with a preset expiry date lasting about 2-3 weeks as well as a few session cookies). I don't see anyone from the EU complaining about GitHub, do you?

I disagree with the implementation because it only targets 127.0.0.1, 10.x.x.x., 192.168.x.x, 172.16.x.x (but not 172.17-31.x.x). It covers most of them (today) but what it things change in future. Instead of trying to outsmart the smart-arses who use something not catered for in this design, how about dropping an IP address-based guess (are we in a LAN or not) with something simpler?

[mandville]

sorry i hit button to early. i am not keen on cookie use.

[sozzled]

Receive users’ consent before you use any cookies except strictly necessary cookies.

Nope. User consent is not stored anywhere.

Waste of time to-ing and fro-ing here when the matter could have been discussed in real-time.

[sozzled]

sorry i hit button to early. i am not keen on cookie use.

Then you aren't keen on using J! (or GitHub) as a logical follow-on from that.

[RussW]

OMFG! I am so sorry that made a mistake and forgot that the 172. range is wider that just 172.16.x.y, yet not once in your disprovals/disagreements did you ever prompt, remind or mention it until now and then "spit the dummy"...

I'll will have to find a 'cuter' method to search through an array of all these ranges, that can then also be added to at a later date, if any more addresses become reserved or need to change.

[sozzled]

Here's an even easier way to check if the age of the "fpa-en.php" file is old:

1) On first activation of the FPA on a website, look for the existence of a file called "fpa-en.ini". 2) If this file doesn't exist, create it (the "last modified date" will be now) in the same folder as where fpa-en.php lives. 3) If the file exists, proceed through the rest of the script. 4) Check the last modified date of fpa-en.ini and, if it's old, delete both that file and the fpa-en.php file if the auto self-destruct conditions are met. 5) The file fpa-en.ini doesn't have to contain anything but it could be used to store run-time settings (in a future version release) if needed.

It's not 100% bullet-proof but it would probably resolve the discussion we've been having.