search

undo-ransomware / ransomware_detection

:arrows_counterclockwise: Ransomware recovery app for Nextcloud
https://apps.nextcloud.com/apps/ransomware_detection
GNU Affero General Public License v3.0
22 stars 6 forks source link

A simple recovery led to complete deletions #48

Open chadwellak opened 3 years ago

chadwellak commented 3 years ago

Testing this, after seeing a BIG RED notification that there were issues (there weren't -- these were nothing but versions issues with automated image uploads), I clicked on A SINGLE ISSUE.

After that, ALL OF MY FILES WERE DELETED. All of the way. I'm talking a 'rm -rf' in the actual filesystem.

ALL of the files for the current user on this server were rm'd. No trash, just gone. Believe it. On the actual filesystem.

Thankfully my local files on the syncing machine were unaffected (thanks, developers. Good f&%*ing job!). It only took a full day and a half to recreate my online folders.

It's pretty hard to believe that this sort of app is even available for trying without this HUGE caveat.

ilovemilk commented 3 years ago

Hi,

I'm really sorry that happend to you. I personally use this application on my Nextcloud instance and tested the version 0.9.0 with multiple ransomware families and also on normal file operations over the time of 3 months.

Nevertheless, this should not happen I deleted this app release from the app store and I will add a warning for all users until this problem is resolved!

To help me resolve this problem can you give me more details how to reproduce this issue?

chadwellak commented 3 years ago

I still need to examine my logs. "Deleted files" from my user space is inaccessible. There seems to be some db corruption.

The files detected were a couple of images (photos) that were auto-uploaded and had a date conflict. This happens sometimes. But I thought I would click on the ransomware recovery to see where the dialogs would take me. They didn't take me anywhere and going back to "All Files" showed that everything was missing.

By "actual filesystem" I mean that examining storage (by looking at the structure of the filesystem on the drive that houses the actual files) reveals that everything has been removed (or possibly just moved, which I'm still holding out hope for).

I am running version 19.0.5.

On Sat, Dec 12, 2020 at 4:11 AM Matthias notifications@github.com wrote:

Hi,

I'm really sorry that happend to you. I personally use this application on my Nextcloud instance and tested the version 0.9.0 with multiple ransomware families and also on normal file operations over the time of 3 months.

Nevertheless, this should not happen I deleted this app release from the app store and I will add a warning for all users until this problem is resolved!

To help me resolve this problem can you give me more details how to reproduce this issue?

  • What files were detected? What version issues did you have with the uploaded images?
  • Are there any hints in your log why your files got wiped? I use the Nextcloud API to delete and recover files so all files should be moved to the trash bin.
  • What do you mean by "in the actual filesystem"? My app doesn't access the filesystem directly the delete process is handled by the Nextcloud API.

— You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub https://github.com/undo-ransomware/ransomware_detection/issues/48#issuecomment-743754155, or unsubscribe https://github.com/notifications/unsubscribe-auth/AF66GYWPHAYHFOPHN5PUN5DSUNTZDANCNFSM4UYAIFAA .

ilovemilk commented 3 years ago

Thanks for reporting back.

Even with a corrupted database the files should still exist on the drive.

Did you already check the trashbin folder in the actual filesystem? You can find it in data/<username>/files_trashbin.

If I can assist you with taking a look at the logs you are always welcome to post them here.

chadwellak commented 3 years ago

I've done a complete search, they're gone. It might be possible to retrieve something with gparted, maybe I'll try. But there were very few files that only existed on the server and, for the most part, won't be missed -- at first anyway.

In any case, thanks for responding. If nothing else this spurs me to implement the backup system.

Attached is a snippet of the nextcloud log at the time this happened.

On Sat, Dec 12, 2020 at 12:58 PM Matthias notifications@github.com wrote:

Thanks for reporting back.

Even with a corrupted database the files should still exist on the drive.

Did you already check the trashbin folder in the actual filesystem? You can find it in data//files_trashbin.

If I can assist you with taking a look at the logs you are always welcome to post them here.

— You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub https://github.com/undo-ransomware/ransomware_detection/issues/48#issuecomment-743910942, or unsubscribe https://github.com/notifications/unsubscribe-auth/AF66GYVHOXIBI6Y7ANEOQX3SUPRORANCNFSM4UYAIFAA .

chadwellak commented 3 years ago

Just to add, if it helps, when this happened all of the folders in my user's root directory were deleted, except one.

This directory was different from all of the others in that it was empty.

On Sat, Dec 12, 2020 at 6:24 PM Erik emc986@gmail.com wrote:

I've done a complete search, they're gone. It might be possible to retrieve something with gparted, maybe I'll try. But there were very few files that only existed on the server and, for the most part, won't be missed -- at first anyway.

In any case, thanks for responding. If nothing else this spurs me to implement the backup system.

Attached is a snippet of the nextcloud log at the time this happened.

On Sat, Dec 12, 2020 at 12:58 PM Matthias notifications@github.com wrote:

Thanks for reporting back.

Even with a corrupted database the files should still exist on the drive.

Did you already check the trashbin folder in the actual filesystem? You can find it in data//files_trashbin.

If I can assist you with taking a look at the logs you are always welcome to post them here.

— You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub https://github.com/undo-ransomware/ransomware_detection/issues/48#issuecomment-743910942, or unsubscribe https://github.com/notifications/unsubscribe-auth/AF66GYVHOXIBI6Y7ANEOQX3SUPRORANCNFSM4UYAIFAA .

ilovemilk commented 3 years ago

Thanks for giving more details but I can't find the attached log file.

Would be great if you can provide it! :)

chadwellak commented 3 years ago

[image: image.png] There's no indication of this attachment? No paperclip at all?

That's odd. If it's really not there, here's a link to grab it:

https://dougflat.ddns.net/s/DBYy8mfTCYmSsGZ

And an online formatter, just in case:

https://www.freeformatter.com/json-formatter.html#ad-output

On Sun, Dec 13, 2020 at 2:06 AM Matthias notifications@github.com wrote:

Thanks for giving more details but I can't find the attached log file.

Would be great if you can provide it! :)

— You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub https://github.com/undo-ransomware/ransomware_detection/issues/48#issuecomment-743990381, or unsubscribe https://github.com/notifications/unsubscribe-auth/AF66GYQDPEHKHWM7K6O3DILSUSN33ANCNFSM4UYAIFAA .

ilovemilk commented 3 years ago

Thanks. I didn't check my mails I use the Github website.

The logs are very helpful. Somehow there was an file operation entry in the database which was null this led to a delete call to "/" which is the root folder of the user. I will add a check if there is a call to "/".

The question which arises is: Why is there an file operation entry which is null? Sadly, there is no hint in the log file why there was a null entry created. This should lead to an exception but I will also add an illegal argument check.

chadwellak commented 3 years ago

I did see that delete call to /. Not having explored your code I didn't want to assume that you hadn't created a virtual fs with its own root, for instance. But that's really good to know. Though it is a mystery to me why that empty folder remained untouched.

I'll have to do some db exploration next I have time and see if I can determine why there is a null operation entry -- maybe the remaining empty folder is a clue.

Thanks for looking into it.

On Sun, Dec 13, 2020 at 10:38 AM Matthias notifications@github.com wrote:

Thanks. I didn't check my mails I use the Github website.

The logs are very helpful. Somehow there was an file operation entry in the database which was null this led to a delete call to "/" which is the root folder of the user. I will add a check if there is a call to "/".

The question which arises is: Why is there an file operation entry which is null? Sadly, there is no hint in the log file why there was a null entry created. This should lead to an exception but I will also add an illegal argument check.

— You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub https://github.com/undo-ransomware/ransomware_detection/issues/48#issuecomment-744057509, or unsubscribe https://github.com/notifications/unsubscribe-auth/AF66GYRXJCK7HEW446GZD4LSUUJ4PANCNFSM4UYAIFAA .

chadwellak commented 3 years ago

Hey, I just wanted to drop another line and apologize if I came across poorly in my initial post to you. I appreciate your prompt responses.

The mystery folder is apparently a folder that the latest edition of Talk populates into a user's /, so that likely means nothing.

But, as I said, I appreciate your help. I've been exploring ways to circumvent ransomware attacks myself. My vector is potentially to do a sanity check but which would also require a backup. And something not specifically tied into a Nextcloud instance. Though I think that using the versioning is a pretty cool idea, php is not my strong suit. But, and I don't want you to take this the wrong way, it seems to me that allowing for a function in your code that provides a permission-free way to recursively remove directories is something that should be inherently avoided. Just my opinion.

In any case, good luck and thanks for your time.

Erik

On Sun, Dec 13, 2020 at 12:40 PM Erik emc986@gmail.com wrote:

I did see that delete call to /. Not having explored your code I didn't want to assume that you hadn't created a virtual fs with its own root, for instance. But that's really good to know. Though it is a mystery to me why that empty folder remained untouched.

I'll have to do some db exploration next I have time and see if I can determine why there is a null operation entry -- maybe the remaining empty folder is a clue.

Thanks for looking into it.

On Sun, Dec 13, 2020 at 10:38 AM Matthias notifications@github.com wrote:

Thanks. I didn't check my mails I use the Github website.

The logs are very helpful. Somehow there was an file operation entry in the database which was null this led to a delete call to "/" which is the root folder of the user. I will add a check if there is a call to "/".

The question which arises is: Why is there an file operation entry which is null? Sadly, there is no hint in the log file why there was a null entry created. This should lead to an exception but I will also add an illegal argument check.

— You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub https://github.com/undo-ransomware/ransomware_detection/issues/48#issuecomment-744057509, or unsubscribe https://github.com/notifications/unsubscribe-auth/AF66GYRXJCK7HEW446GZD4LSUUJ4PANCNFSM4UYAIFAA .

ilovemilk commented 3 years ago

Hey, I completely understand your reaction, I would have reacted the same way. This should not happen.

I want to help the people to have safe and easy way to deal with such problems and I try to react as fast as possible.

My vector is potentially to do a sanity check but which would also require a backup.

I think we might have the same idea here. I think it's nearly impossible to detect a ransomware attack in realtime if the ransomware family is unknown and it's necessary to gather as much data of the behaviour as possible but by gathering such data we lose data. Thus the only real option is to use a backup, gather as much data as possible, identify the attack pattern and then revert the attack afterwards.

And something not specifically tied into a Nextcloud instance.

I choose Nextcloud because it has all the necessary functionality built in (backup and file versioning) but I think it's also possible to do something similar with other backups.

But, and I don't want you to take this the wrong way, it seems to me that allowing for a function in your code that provides a permission-free way to recursively remove directories is something that should be inherently avoided. Just my opinion.

I agree with you but I have to state this was a fault in my implementation. Normally, the recover function should revert the operations one by one that the integrated Nextcloud file versioning is able to treat them the right way.

Sadly, the problem you discovered is a combination of missing sanity checks which got lost during my refactoring. Which are:

This led to the deletion of the users files folder which is not covered by the Nextcloud file versioning or backup and is also not protected by special permissions like the root folder.

If you have any ideas or comments on how to improve this any further I am always happy to hear them.

andros-ua commented 3 years ago

All my files on filesystem were deleted also. Thanks. And i found out about this problem right after looking to it today. Now I'm really in desperate, because a lot of data which i didn't backed up yet was gone in one moment.

ilovemilk commented 3 years ago

I'm very sorry to hear that. I removed the release and added a warning as soon as the problem arise. Sadly, Nextcloud doesn't offer a way to disable or remove the app for all users. I want to reach all users to warn them not use this app but I can't.

I now removed all releases from the app store and added a new version only containing a warning. I hope this reaches more people because there is nothing more I can do.

I tried to resolve this problem but couldn't reproduce the problem at all. So I'm currently reworking the whole app but I currently tend to remove the app completely because of such problems which I can't reproduce but harm peoples data.

jospoortvliet commented 3 years ago

Hi @ilovemilk I was wondering if you have any news on the rework of the app - it is rather sad that we're all missing it now ;-)

ilovemilk commented 3 years ago

Hi @jospoortvliet I found the critical bug but I work on a concept to test the app in a way that this can't happen again.

Normally I do this in the following way:

But after this critical bug I am a little scared that it could happen again and as I don't want to harm anyones data I want to improve it this further. I want to rework the app code to be more unit testable because some parts are missing tests and I am thinking about a way to run some integration tests with ransomware samples automatically.