invalid multibyte escape

[ash68]

hi, running "piggly trace" gives me the below error message repeatedly. is it because i am using $a$ in some postgresql functions?

ProcessQueue running concurrently /mnt/hgfs/g/kputnam-piggly-7560eb8/lib/piggly/compiler/cache_dir.rb:20: invalid multibyte escape: /[\000-\010\016-\037\177-\300]/ /mnt/hgfs/g/kputnam-piggly-7560eb8/lib/piggly/compiler/trace_compiler.rb:22:in compile' /mnt/hgfs/g/kputnam-piggly-7560eb8/lib/piggly/command/trace.rb:49:inblock (2 levels) in trace' /mnt/hgfs/g/kputnam-piggly-7560eb8/lib/piggly/util/process_queue.rb:79:in call' /mnt/hgfs/g/kputnam-piggly-7560eb8/lib/piggly/util/process_queue.rb:79:inblock in concurrently' /mnt/hgfs/g/kputnam-piggly-7560eb8/lib/piggly/util/process_queue.rb:77:in fork' /mnt/hgfs/g/kputnam-piggly-7560eb8/lib/piggly/util/process_queue.rb:77:inconcurrently' /mnt/hgfs/g/kputnam-piggly-7560eb8/lib/piggly/util/process_queue.rb:47:in execute' /mnt/hgfs/g/kputnam-piggly-7560eb8/lib/piggly/command/trace.rb:53:intrace' /mnt/hgfs/g/kputnam-piggly-7560eb8/lib/piggly/command/trace.rb:32:in main' /mnt/hgfs/g/kputnam-piggly-7560eb8/lib/piggly/command/base.rb:15:inmain' /mnt/hgfs/g/kputnam-piggly-7560eb8/bin/piggly:8:in `

'

[kputnam]

Thanks for reporting this. It is a bug in my code, not yours. The cause was a regular expression that was intended to match single-byte characters. Since Ruby 1.9, different character encodings are supported, and the regular expression happens to denote invalid combinations of bytes in that case.

This should be fixed now, but I don't have an environment where I can test it (unfortunately the unit tests don't cover this either). Let me know if you still have problems.

[dxlbnl]

@kputnam : I don't think its entirely fixed.

I have a database with some complex queries, not sure the error might be correlated with that. This is the error I'm getting: https://gist.github.com/neoel/c5d63a4839f6f557487b1639ec4416e9

And here you can find the sql defining the functions: https://github.com/wolbodo/pms/blob/master/database/db-logic.sql

I'm using:

$ ruby -v
ruby 2.3.0p0 (2015-12-25) [x86_64-linux-gnu]
$ rake --version
rake, version 11.1.2
$ gem -v
2.5.1

[kputnam]

I can't reproduce the error, but I definitely see it. If you edit lib/piggly/parser/parser.rb and put this as the very first line, does that error about invalid multibyte sequence go away (you may get other errors)?

# encoding: ASCII-8BIT

[kputnam]

Did this work, @neoel, or did you need any help trying it out?

[dxlbnl]

I think it might have worked, but since I can't run my personal project I'm not 100% sure

Dexter - DexterLabs.nl

On Thu, May 19, 2016 at 7:57 AM, Kvle Putnam notifications@github.com wrote:

Did this work, @neoel https://github.com/neoel, or did you need any help trying it out?

— You are receiving this because you were mentioned. Reply to this email directly or view it on GitHub https://github.com/kputnam/piggly/issues/23#issuecomment-220234345

[Russ-K]

Hi, I get this issue when trying to use v1.2.1. I'm using ruby 2.1.0 on Windows 7 x64.

I've added the following line:

encoding: ASCII-8BIT

to the top of the parser.rb file but it makes no difference. However, adding it to the top of the cache.rb file makes the error go away but results in the "piggly" doing literally nothing. I can run "piggly hello world" and it doesn't complain at all.

HTH

[kputnam]

Thanks @RussellKeane. I think that might have fixed the encoding issue, but maybe you're not seeing any output due to an issue with fork on Windows. I left a comment in #22 about that -- feel free to get in touch by email or just leave a comment if you're interested in helping with that!

[kputnam]

@RucialK now that you've fixed the Windows compatibility issue, is this still broken?

[cchavez]

As work around I changed the the file lib/piggly/parser.rb:

line 22: tree = p.parse(input) to tree = p.parse(input.force_encoding("ASCII-8BIT")) line 26: input.replace(string) to input.replace(string.force_encoding("ASCII-8BIT"))

but now I'm getting an error that it seems to be related to the pg driver:

lib/piggly/installer.rb:52:in `exec': ERROR: does not exists the type int4[] (PG::UndefinedObject)

my stored procedure contains an array of integers.

create or replace function "public"."actualizarinatecdetalleplanilla" ("int4", "int4", "int4", "int4[]") returns "pg_catalog"."int4" ...

[kputnam]

@cchavez That seems like a strange error, what version of PostgreSQL are you using? Just to be sure, can you run this (it should work)?

create function f(int4[]) returns int4[] as 'begin return $1; end' language plpgsql;
drop function f(int4[]);

[cchavez]

That code works find.

Not sure what is the error with my function.

I'm using postgresql 9.5.5. I also updated the pg gem to the latest the pg (0.19.0).

I also got some errors with some void functions, but I tested with the sample you sent me and it works some void function as well.

[cchavez]

I think I found the issue, when the source code is generated is quoting the data types so for example it generate:

create or replace function "public"."actualizaringrfijosdetalleplanilla" ("int4", "int4", "int4", "int4[]")

noticed the "int4" does not bother to prostgresql, but "int4[]" it does not like to postgresql, when I remove the quote and use just int4[] works as expected.

The quote happen at lib/piggly/dumper/qualified_type.rb

[cchavez]

Other issue I found it was in the parser the following sentence shows an error:

IF (x>0)THEN ....

noticed that there is not blank between the ) and THEN, but that is okay to postgresql.

[cchavez]

Hi any idea about the following error messages when generate the report?

/lib/piggly/profile.rb:25:in `[]': No tag with id 65f387aaca4e7553 (RuntimeError)

[kputnam]

You might need to reset things; that error can happen if the instrumented code running in the database is not in sync with the code piggly thinks is in the database.

I think you can do that with piggly untrace to restore the original source code, then move aside any cache files (off the top of my head, I forget where piggly places these), then run piggly trace again.

[cchavez]

Right, thanks. I trace again over a fresh database an generate the report repeat the trace and I was able to retrieve the report.

[cchavez]

@kputnam how I can add at the beginning if the generated file "parser.rb" the following:

encoding: ASCII-8BIT

When I run the command from the source code it works, but when I run the command from the built gem it throws the error:

piggly-2.0.0/lib/piggly/parser/parser.rb:5972:in `initialize': invalid multibyte escape: /\A[a-z\200-\377_]/ (RegexpError)

I think that can be fixed if we add that line the generated file "parser.rb"

wdyt?

[kputnam]

I think you're right! Try adding it to lib/piggly/parser/grammar.tt which is used to generate parser.rb. I think changes to grammar.tt should automatically regenerate parser.rb, but if not, you may need to delete it manually to force it to regenerate.

[kputnam]

@cchavez, I'm not sure how I missed your remark about "int4[]" being incorrectly quoted, but I ran against that issue in #31 and have fixed it.