-rcsv option ignored (Trunk)

[GoogleCodeExporter]

What steps will reproduce the problem?
1. Compile from Trunk
2. bafprp -rcsv -prdelimiter:,
-prfields:type,originating_number,terminating_number,date,time,trunk_id
SOME_BAF_FILE

What is the expected output? What do you see instead?

I would expect a csv output.  Instead I get standard output as if no
options were selected.

What version of the product are you using? On what operating system?

bafprp from Trunk on 2009-11-04 (r178 ?)  Tested on FreeBSD 7.2 amd64
Please provide any additional information below.

Original issue reported on code.google.com by th...@bendtel.net on 4 Nov 2009 at 9:43

[GoogleCodeExporter]

did you get any log messages?
The command line looks good so I would expect bafprp to complain if it could 
not find
or could not use csv

Original comment by charless...@gmail.com on 4 Nov 2009 at 9:52

[GoogleCodeExporter]

With this command line:

bafprp -lfile -plfilename:bafprp.log -rcsv -prdelimiter:,
-prfields:type,originating_number,terminating_number,date,time,trunk_id 
SOME_BAF_FILE

I first see this:

Set record output to file
Set error output to file
Set log output to file
Set record property filename to /var/log/bafprp/record.log
Set error property filename to /var/log/bafprp/error.log
Set log property filename to /var/log/bafprp/log.log
Set log output to file
Set log property filename to bafprp.log
Set record output to csv
Set record property delimiter to ,
Set record property fields to
type,originating_number,terminating_number,date,time,trunk_id

This suggests to me it knows what I want...

I then get this in the bafprp.log file I specified:

- 2009-11-04 14:06:17 INFO: Running Program
- 2009-11-04 14:06:17 INFO: Processing SOME_BAF_FILE started
- 2009-11-04 14:06:17 WARN: This record has indicated that it contains errors
- 2009-11-04 14:06:52 INFO: Parsed 156928 records
- 2009-11-04 14:06:52 INFO: Removed 13 duplicates
- 2009-11-04 14:06:52 ERROR: No valid file for record output, falling back to 
console
output
- 2009-11-04 14:12:40 INFO: Processing SOME_BAF_FILE ended
- 2009-11-04 14:12:40 INFO: Cleanup started
- 2009-11-04 14:12:43 INFO: Cleanup ended
- 2009-11-04 14:12:43 INFO: Program Terminated

The output to the console is not CSV, but the output I normally see without 
specifying.

Original comment by th...@bendtel.net on 4 Nov 2009 at 10:16

[GoogleCodeExporter]

That is one mean baf file, 160k records.  Out of curiosity can you tell me how 
much
ram bafprp uses while processing the records? :D

As for your problem, it seems that the csv output could not open
/var/log/bafprp/record.log
There was an issue along these same lines a month ago, I think the problem 
ended up
being a permission thing.
Try adding -prfilename:~/myrecords.log to the end there.  If it still can't 
open the
file let me know.

Original comment by charless...@gmail.com on 4 Nov 2009 at 11:14

[GoogleCodeExporter]

Interesting that you ask about the RAM ;)  I filed an issue about not having 
enough
on my main workstation to process this file.  On the machine it works on, it 
seems to
use about 1295M before it creates output.

Original comment by th...@bendtel.net on 4 Nov 2009 at 11:31

[GoogleCodeExporter]

I did as you suggested, initial output was:

Set record output to file
Set error output to file
Set log output to file
Set record property filename to /var/log/bafprp/record.log
Set error property filename to /var/log/bafprp/error.log
Set log property filename to /var/log/bafprp/log.log
Set log output to file
Set log property filename to bafprp.log
Set record property filename to ~/myrecords.log
Set error property filename to ~/myerrors.log
Set record output to csv
Set record property delimiter to ,
Set record property fields to
type,originating_number,terminating_number,date,time,trunk_id
Set record output to console

Still not getting CSV output.  My gut tells me there is probably something very 
minor
in the code preventing this, but I haven't yet spent the time to get a handle 
on the
source.

Original comment by th...@bendtel.net on 4 Nov 2009 at 11:57

[GoogleCodeExporter]

I will try and reproduce your results but last I checked bafprp was outputing 
csv to
files.  
For some reason it seems like your bafprp cannot create the file it needs to 
write
to, very strange.

Original comment by charless...@gmail.com on 5 Nov 2009 at 7:22

• Changed state: Accepted
• Added labels: OpSys-Linux, Usability

[GoogleCodeExporter]

I made some updates to the csv output, please verify if the new trunk version 
works
for you.

Original comment by charless...@gmail.com on 17 Nov 2009 at 5:43

• Changed state: Fixed

[GoogleCodeExporter]

This is still not quite working for me as expected.

I downloaded and compiled new code as of today.

Command line I am trying is: bin/bafprp -lfile -plfilename:bafprp.log -rcsv
-prdelimiter:,
-prfields:type,originating_number,terminating_number,date,time,trunk_id 
MY_BAF_FILE

It appears to know exactly what I want, because the output starts:

Set record output to file
Set error output to file
Set log output to file
Set record property filename to /var/log/bafprp/record.log
Set error property filename to /var/log/bafprp/error.log
Set log property filename to /var/log/bafprp/log.log
Set log output to file
Set log property filename to bafprp.log
Set record output to csv
Set record property delimiter to ,
Set record property fields to
type,originating_number,terminating_number,date,time,trunk_id
Set record output to console

But then it leads right into default output.

If I specify a command line like: bin/bafprp -rfile -lfile -efile
-prfilename:output.csv -pefilename:error.log -plfilename:bafprp.log -rcsv
-prdelimiter:,
-prfields:type,originating_number,terminating_number,date,time,trunk_id BAF_FILE

...where I have specified an output file, then I *do* get delimited output.  
However,
it appears that my delimiter argument was ignored because a ";" was used 
instead of a
",".

Original comment by th...@bendtel.net on 19 Nov 2009 at 7:09

[GoogleCodeExporter]

Also, I just noticed as I looked at the output that the requested fields
originating_number, terminating_number, and trunk_id do not print at all in any 
of
the records.  A sample of the output follows:

Record 625: Incoming CDR;;;2009-09-30;08:25:55;;
Record 1: Local message rate call;;;2009-09-30;08:25:55;;
Record 1: Local message rate call;;;2009-09-30;08:29:32;;
Record 1: Local message rate call;;;2009-09-30;08:28:56;;
Record 1: Local message rate call;;;2009-09-30;08:27:10;;
Record 625: Incoming CDR;;;2009-09-30;08:27:10;;
Record 625: Incoming CDR;;;2009-09-30;08:29:39;;
Record 625: Incoming CDR;;;2009-09-30;08:28:49;;
Record 1: Local message rate call;;;2009-09-30;08:28:49;;
Record 1: Local message rate call;;;2009-09-30;08:29:12;;
Record 625: Incoming CDR;;;2009-09-30;08:29:15;;
Record 1: Local message rate call;;;2009-09-30;08:28:42;;
Record 625: Incoming CDR;;;2009-09-30;08:26:55;;
Record 625: Incoming CDR;;;2009-09-30;08:29:21;;
Record 625: Incoming CDR;;;2009-09-30;08:29:07;;
Record 1: Toll call or non-US;;;2009-09-30;08:29:44;;
Record 625: Incoming CDR;;;2009-09-30;08:29:17;;
Record 625: Incoming CDR;;;2009-09-30;08:25:35;;
Record 1: Local message rate call;;;2009-09-30;08:28:40;;
Record 625: Incoming CDR;;;2009-09-30;08:28:58;;
Record 1: Toll call or non-US;;;2009-09-30;08:29:54;;
Record 625: Incoming CDR;;;2009-09-30;08:28:23;;
Record 1: Local message rate call;;;2009-09-30;08:26:45;;
Record 625: Incoming CDR;;;2009-09-30;08:29:18;;
Record 625: Incoming CDR;;;2009-09-30;08:29:14;;
Record 1: Local message rate call;;;2009-09-30;08:29:07;;
Record 625: Incoming CDR;;;2009-09-30;08:29:36;;
Record 1: Local message rate call;;;2009-09-30;08:29:36;;
Record 625: Incoming CDR;;;2009-09-30;08:28:44;;
Record 1: Local message rate call;;;2009-09-30;08:29:49;;
Record 625: Incoming CDR;;;2009-09-30;08:29:45;;
Record 625: Incoming CDR;;;2009-09-30;08:28:54;;
Record 625: Incoming CDR;;;2009-09-30;08:26:17;;
Record 625: Incoming CDR;;;2009-09-30;07:43:35;;
Record 1: Local message rate call;;;2009-09-30;08:28:52;;
Record 625: Incoming CDR;;;2009-09-30;08:29:13;;
Record 1: Local message rate call;;;2009-09-30;08:29:23;;

Original comment by th...@bendtel.net on 19 Nov 2009 at 7:17

[GoogleCodeExporter]

You got some field names incorrect there, 
I believe the fields you are looking for are
-prfields:type,originatingnpa,originatingnumber,terminatingnpa,terminatingnumber
,date,time,trunkid

originating number and terminating number used to be one field but I found out 
while
testing with different baf files that there is a reason why the GR-1100 
specifies the
fields separately.  So I divided up the number.

Original comment by charless...@gmail.com on 21 Nov 2009 at 5:35

[GoogleCodeExporter]

Ah, I see about the fields.  I had used an example I found in the wiki under 
Fields.
 I guess things have changed since then.  Still my confusion remains about the ","
being ignored.  Is it because there are possible commas in the output?  Also, 
why
does it not do delimited if you don't specify an output file?

Original comment by th...@bendtel.net on 23 Nov 2009 at 6:20

[GoogleCodeExporter]

Also, it always seems to end with a delimiter character.  Normally this would
indicate an empty final field.

Original comment by th...@bendtel.net on 23 Nov 2009 at 6:22

[GoogleCodeExporter]

I just commited some code that should hopefully trim off the last delimiter for
record output.
And your ignored delimiter problem should now be solved, there was a problem 
with my
spelling.
Unfortunately due to thanksgiving I am once again at a location without any pri 
files
so I cannot test it yet, but I will give it a whirl as soon as I can.

Once this issue gets ironed out I will make up a new 1.3 release of bafprp.

Original comment by charless...@gmail.com on 27 Nov 2009 at 2:11

[GoogleCodeExporter]

Awesome.  I'll try to do some testing this weekend.  Here is a 
Makefile_BSD_NoSQL
based on the tree I just grabbed via SVN (today, Thanksgiving, 2009-11-26).  
This
should work with both FreeBSD and OpenBSD.  For OpenBSD, there is one line that 
needs
to be uncommented (which is specified by a comment in the file).  If this gets
included with the distribution, I will make some notes in the wiki.

Original comment by th...@bendtel.net on 27 Nov 2009 at 6:40

Attachments:

• Makefile_BSD_NoSQL

[GoogleCodeExporter]

Did the updated makefiles still not work for you?
I see you replaced all the ()'s with {}'s and put those for loops in.

I think the new makefile should fix it so you dont need the for loops, but if 
BSD
wants {}'s instead of ()'s you might still have some problems.

Original comment by charless...@gmail.com on 27 Nov 2009 at 3:53

[GoogleCodeExporter]

It does work (on OpenBSD) if I add "-lm" to LDFLAGS.  That was not needed on 
FreeBSD,
so it may work as is on there.  The real reason I made a new Makefile is 
because the
ones provided require gmake, which is not standard equipment on *BSD.  Also, I 
added
the -lm on a seperate line with a comment, but that's not a big deal if that 
gets
documented in some way (I was going to add it to the wiki).

So, if you want to stick to the ones provided, I could just put some notes in 
the
wiki.  I could even link to my Makefile if BSD folks want to download it.

Original comment by th...@bendtel.net on 27 Nov 2009 at 6:29

[GoogleCodeExporter]

ahh I see now.

I can include the bsd makefile no problem.
I wonder why openbsd thinks bafprp needs the math library, interesting.

Original comment by charless...@gmail.com on 27 Nov 2009 at 6:49

[GoogleCodeExporter]

If you don't specify the math lib, you get errors like this:

/usr/lib/libstdc++.so.45.0: undefined reference to `log'
/usr/lib/libstdc++.so.45.0: undefined reference to `sqrt'
/usr/lib/libstdc++.so.45.0: undefined reference to `cosh'
/usr/lib/libstdc++.so.45.0: undefined reference to `finitef'
/usr/lib/libstdc++.so.45.0: undefined reference to `cos'
/usr/lib/libstdc++.so.45.0: undefined reference to `tanh'
/usr/lib/libstdc++.so.45.0: undefined reference to `sin'
/usr/lib/libstdc++.so.45.0: undefined reference to `atan2'
/usr/lib/libstdc++.so.45.0: undefined reference to `pow'
/usr/lib/libstdc++.so.45.0: undefined reference to `sinh'
/usr/lib/libstdc++.so.45.0: undefined reference to `log10'
/usr/lib/libstdc++.so.45.0: undefined reference to `exp'
/usr/lib/libstdc++.so.45.0: undefined reference to `tan'

Original comment by th...@bendtel.net on 27 Nov 2009 at 7:24

[GoogleCodeExporter]

Setting this back to accepted since csv still outputs the trailing delimiter

Original comment by charless...@gmail.com on 10 Dec 2009 at 5:10

• Changed state: Accepted

[GoogleCodeExporter]

Trailing delimiter issue appears fixed for me in svn as of 2009-12-10.

Original comment by th...@bendtel.net on 10 Dec 2009 at 10:14

[GoogleCodeExporter]

Indeed you are correct, guess i should make sure that a bug exists before 
claiming it
does.  Although I could have sworn I saw a comment to that effect last week..

Original comment by charless...@gmail.com on 11 Dec 2009 at 12:05

• Changed state: Verified