Add logging to all user actions

[leomurta]

We need some kind of logging of which used did what, especially when the user action changes the database.

Are there some gem to help on this?

[tiagoamaro]

I used a gem in a project that was useful for tracking a certain model history. The gem is called paper_trail, and I found it quite resourceful.

You could check on the Active Record Versioning category on Ruby Toolbox for more gems :)

[JoaoFelipe]

I tried to use the paper_trail, but I'm getting an error

I can't open the server to see the error right now, but it is something like "no such table main.items. INSERT INTO versions (..." By using the sqlite shell, I can see the table 'versions'. It seems to be a problem with the name main.items, that is probably used by the paper_trail

Do you know if there is anything that could cause conflicts?

[lvieirajr]

I got the same error that JoaoFelipe got, when I tried using paper_trail on SAPOS, after that, I tried using the paper_trail gem on a newly created project, and it worked just fine. So, it seems that this is a conflict with some configuration or even with another gem.

We have some options: 1- We can fork the paper_trail gem to try and fix the conflict 2- We can try to find the conflict and post it on the paper_trail github so that someone will fix it. ( might take a while ) 3- We can find another gem that does the same thing 4- We can create our own logging system

Edit: Tried another gem called Audited (-v 3.0), and I get pretty much the same error, but instead of main.items that is the table for the paper_trail I get the error on the main.associateds that is the table for the Audited gem.

[leomurta]

I do not like options 1 and 4 because they impose maintenance efforts over us. It is always better to use popular gems (that have active development) in its most recent stable release.

[tiagoamaro]

@JoaoFelipe and @LuisJR , could you please put in a Gist the complete error log generated by this error?

I had an error in a project that needed versioning of a specific model and ran into some problems too. I searched the paper_trail issue tracker and found a solution to my problem.

The paper_trail maintainers are very active and have a good Opened/Closed issue ratio (12 opened / 261 closed).

[lvieirajr]

I Created a new issue for the paper_trail https://github.com/airblade/paper_trail/issues/275

I don't know how well it is explained and if I did it the right way, don't know if there are any how-tos on creating issues. Hopefully they will be able to help us.

[tiagoamaro]

@JoaoFelipe and @LuisJR , did you go through paper_trail's installing process? When did this error happened?

Paper trail requires an extra table where it registers its versioning. By requiring a table, it needs a migration (That is generated by the generator created by the gem) and afterwards, running the command rake db:migrate.

Check here in the installation session of 2.7 version (stable)

[lvieirajr]

Yes we did. The error happens when we try to save any model that uses the has_paper_trail. If we check the database the table versions (the one that the gem migration creates) is in there, and everything goes just fine with the installation.

The thing is, when we try to save the data, it asks for a table main.items that we couldn't find any information about.

[tiagoamaro]

@LuisJR , The problem was with the schema being generated by the migrations.

SAPOS has a gem called schema_plus that creates automatic foreign keys on migrations based on its _id elements.

paper_trail creates a table with an item_id attribute, and combined with schema_plus, the resulting schema references a table named items, that does not exist.

Make sure to delete the line on the schema that references the items table after its migration, and paper_trail should work.

You can close the issue on paper_trail repository, warning them that the gem schema_plus may lead to incompatibilities :)

[lvieirajr]

Thanks @tiagoamaro, this comment really solved the problem.

The only problem is that I had to do a little bit more, besides removing the line on the schema, I also had to drop the table versions and recreate it on the SQLite console, because with the migration it would always recreate the foreign key.

[lvieirajr]

Did a little bit more of research and I found that the schema_plus gem has the option to turn of the foreign_key creation.

What I did was, after generating the migration using the command: bundle exec rails generate paper_trail:install

I went to the migration file and added ", foreign_key: false" to the end of the line of the item_id.

Now it creates the table without the foreign key, so we won't have to manually create it.

[lvieirajr]

Here is what the log screen looks like right now: You have 2 links, one of them takes you to the user who executed that action, and the other one takes you to the object visualization (as it is in the database at this point)

Here is what the show link contais for now: I'm still working on formatting the details of the object as it was before the action was executed

[leomurta]

Great!

What about changing the section name from "Versões" to "Mudanças" or "Log"? The tuples actually represent changes, not versions (they contain versions of objects). Moreover, is it possible to have the change date/time in the first screen?

Regarding the second screen, it would be nice to have both previous and next versions of the object shown (like you do to the previous version, it would be nice to "open" the next version). Actually, what about diffing them? Is it possible? There are any gems for diffing? This can become a new issue to not postpone the release of this one. If you cannot find a gem available, we can code a diff algorithm based on LCS. It is pretty easy.

Finally, is "versão atual" a correct name? I mean, is it really the current version or the next version?

What about writing down (in portuguese) a bit, showing the way things are stored in the DB, the concepts used in this approach, and an example of how history is tracked? This would help people to understand how it work and can be used as a section of your term paper. Btw, this could also migrate to SAPOS manual as soon as we release.

[lvieirajr]

@leomurta I added the created at column, so now it shows when the action happened. Also changed the name to Log

The "versão atual" is the one that is in the database at the moment that the person is looking at it. Even if you change that object more than once, and if you click on the link at the "versao atual" you will always go to the same page, that is the page of that object that is currently in the database.

Edit:"" The version that you call "next" is the one that is already showing as a link on the second screen, and the unformatted data on the bottom is the "old" version. "" And I'll try to write later tonight or maybe tomorrow, a more in-depth "guide" to how things are working behind all this.

[leomurta]

Is it possible to capture the version produced by the change? I mean, assume name="John", but I changed it to "Peter" and you changed it to "Carl" after my change. It would be nice to have something like this:

User: Leo Old: "John" New: "Peter"

User: Luis Old: "Peter" New: "Carl"

As you explained, the data is in the database, so it seems some navigation would suffice to capture these states...

[lvieirajr]

It is possible.

I don't know how easily that could be done, but it's certainly possible.

I only see one problem, that is, we are not able to "open" the versions that are not the ones currently in the database. Using your example. After we changed the object name to Carl, we are not able to open a users/id window for John or Peter, because they don't exist anymore. So All we would be able to show is the information that we are showing now, on the "Versao Antiga" section, as shown below.

[leomurta]

Lets release this feature as is and open a new issue to try to improve this. It seems you can run a query to get the next change over the same object (of course, if timestamp is available). With this, you could get the "versão antiga" of the next object, which in fact is the next version that we are looking for. When there are no such next change, the next version is the current version.

But, as I said, the information is in the DB somehow, and this is more than fine to release as is.

[lvieirajr]

closed #60

[braganholo]

When I tryed to open the log page, I got this error: The page you were looking for doesn't exist. You may have mistyped the address or the page may have moved.

[JoaoFelipe]

Did you tested this issue after the last deploy? I think it was fixed, but I forgot to close