search

br1ghtyang / asterixdb

Automatically exported from code.google.com/p/asterixdb
0 stars 0 forks source link

No value parser factory for delimited fields of type DATE #54

Open GoogleCodeExporter opened 8 years ago

GoogleCodeExporter commented 8 years ago 
Hi Raman,

This test was ignored from runtime tests group, when it is run individually it 
fails.

Failing test case : src/test/resources/runtimets/queries/scan/temp_types_02.aql

khurram@ubuntu:~/google-code/trunk/asterix$ svn info
Path: .
URL: http://asterixdb.googlecode.com/svn/trunk/asterix
Repository Root: http://asterixdb.googlecode.com/svn
Repository UUID: eaa15691-b419-025a-1212-ee371bd00084
Revision: 29
Node Kind: directory
Schedule: normal
Last Changed Author: vinayakb
Last Changed Rev: 26
Last Changed Date: 2012-03-13 12:17:32 -0700 (Tue, 13 Mar 2012)

khurram@ubuntu:~/google-code/trunk/hyracks_dev_next$ svn info
Path: .
URL: http://hyracks.googlecode.com/svn/branches/hyracks_dev_next
Repository Root: http://hyracks.googlecode.com/svn
Repository UUID: 123451ca-8445-de46-9d55-352943316053
Revision: 1287
Node Kind: directory
Schedule: normal
Last Changed Author: vinayakb
Last Changed Rev: 1274
Last Changed Date: 2012-03-05 23:01:47 -0800 (Mon, 05 Mar 2012)

Test platform :

khurram@ubuntu:~$ uname -a
Linux ubuntu 2.6.38-13-generic #54-Ubuntu SMP Tue Jan 3 13:38:12 UTC 2012 
x86_64 x86_64 x86_64 GNU/Linux

Here are the steps to reproduce the problem,

drop dataverse test if exists;
create dataverse test;
use dataverse test;

create type TempType as closed {
  id: int32,
  date: date,
  time: time,
  datetime: datetime,
  duration: duration
}

create nodegroup group1 on nc1;

create dataset Temp2(TempType)
        partitioned by key id on group1;

load dataset Temp2
using "edu.uci.ics.asterix.external.dataset.adapter.NCFileSystemAdapter"
(("path"="nc1://data/nontagged/tempData.txt"),("format"="delimited-text"),("deli
miter"="|")) pre-sorted;

write output to nc1:"rttest/scan_temp_types_02.adm";

for $a in dataset('Temp2')
return $a

Test hangs when executed from Eclipse.
Here is the trace,
 ***** BEGIN TESTS from file :src/test/resources/runtimets/queries/scan/temp_types_02.aql
0 : Asterix { 
  nts;
}
   Outputs:
      0 : edu.uci.ics.hyracks.dataflow.std.connectors.OneToOneConnectorDescriptor@4ab83be0
1 : 
edu.uci.ics.hyracks.storage.am.common.dataflow.TreeIndexBulkLoadOperatorDescript
or@5631573
   Inputs:
      0 : edu.uci.ics.hyracks.dataflow.std.connectors.OneToOneConnectorDescriptor@4ab83be0

edu.uci.ics.hyracks.api.exceptions.HyracksDataException: initialization of 
adapter failed
    at edu.uci.ics.asterix.external.data.operator.ExternalDataScanOperatorDescriptor.createPushRuntime(ExternalDataScanOperatorDescriptor.java:61)
    at edu.uci.ics.hyracks.control.nc.work.StartTasksWork.doRun(StartTasksWork.java:114)
    at edu.uci.ics.hyracks.control.common.work.SynchronizableWork.run(SynchronizableWork.java:32)
    at edu.uci.ics.hyracks.control.common.work.WorkQueue$WorkerThread.run(WorkQueue.java:115)
Caused by: 
edu.uci.ics.hyracks.algebricks.core.api.exceptions.NotImplementedException: No 
value parser factory for delimited fields of type DATE
    at edu.uci.ics.asterix.external.data.parser.DelimitedDataStreamParser.initialize(DelimitedDataStreamParser.java:53)
    at edu.uci.ics.asterix.external.dataset.adapter.NCFileSystemAdapter.initialize(NCFileSystemAdapter.java:72)
    at edu.uci.ics.asterix.external.data.operator.ExternalDataScanOperatorDescriptor.createPushRuntime(ExternalDataScanOperatorDescriptor.java:59)
    ... 3 more

Regards
Khurram

Original issue reported on code.google.com by khfaraaz82 on 14 Mar 2012 at 5:17

GoogleCodeExporter commented 8 years ago 
Need to support dates to the extent that we can parse date formats in web logs 
etc.

Original comment by vinay...@gmail.com on 6 Apr 2012 at 8:55

GoogleCodeExporter commented 8 years ago 
Jarod, did you fix this in your recent temporal type changes which are now into 
asterix_stabilization ? can you pls confirm, thanks.

Original comment by khfaraaz82 on 19 Nov 2012 at 3:57

GoogleCodeExporter commented 8 years ago 
We have implemented the import feature from adm file, however it has not been 
tested on delimited text. I will verify this in my temporal_functionality 
branch and fix there if not yet.

Original comment by jarod...@gmail.com on 19 Nov 2012 at 4:29

GoogleCodeExporter commented 8 years ago 
Reassign to me to fix it.

Original comment by jarod...@gmail.com on 19 Nov 2012 at 4:30

GoogleCodeExporter commented 8 years ago 
Thanks Jarod.

Original comment by khfaraaz82 on 19 Nov 2012 at 7:14

GoogleCodeExporter commented 8 years ago 
Confirmed as missing features. Will be fixed in the temporal functionality 
branch.

Original comment by jarod...@gmail.com on 20 Nov 2012 at 4:54

GoogleCodeExporter commented 8 years ago 
Seems that this involves changes to hyracks, as the value parsers used in 
NtDelimitedDataTupleParserFactory are directly from Hyracks package. However as 
far as I know other importing parsers, like the one for importing json files, 
are in asterix. I think these two cases should be consistant. I need more 
investigation on this.

Original comment by jarod...@gmail.com on 20 Nov 2012 at 5:07

GoogleCodeExporter commented 8 years ago 
This issue was closed by revision r942.

Original comment by jarod...@gmail.com on 25 Nov 2012 at 7:41

GoogleCodeExporter commented 8 years ago

Original comment by jarod...@gmail.com on 25 Nov 2012 at 11:12

GoogleCodeExporter commented 8 years ago 
Why is this FixedInBranch ?
Is it blocked on something that is stopping it from making it to 
asterix_stabilization ?
If the fix is waiting on a code review, it should not be marked as 
FixedInBranch.

Please update the current status of this issue with comments. Thanks.

Original comment by khfaraaz82 on 19 Jan 2013 at 12:57

GoogleCodeExporter commented 8 years ago 
The fix is in the asterix_stabilization_temporal_functionality branch. The code 
need to be reviewed before being merged back.

Original comment by jarod...@gmail.com on 24 Jan 2013 at 5:37

GoogleCodeExporter commented 8 years ago 
The overall external temporal data importing (including interval, date, time, 
datetime and duration) will be finished after the new lexer is merged.

Original comment by jarod...@gmail.com on 11 Feb 2013 at 4:29

GoogleCodeExporter commented 8 years ago 
Do we still need to support non-primitive types (types except for the int, 
string, double; saying temporal types)? As from Diego's work we now have good 
support for adm (json) data source, delimited external data may be converted 
into the adm before importing?

Original comment by jarod...@gmail.com on 4 Mar 2013 at 11:27

GoogleCodeExporter commented 8 years ago 
We need to support temporal types for sure.   Ji Mahn's work needs it, and 
Eugenia's work can also benefit from it.  I remember once Jarod, Diego, and I 
had a discussion about how to deal with complicated strings in lexer, and 
decided to let the parser interprete the strings.  Is that conclusion related?

Original comment by che...@gmail.com on 5 Mar 2013 at 6:08

GoogleCodeExporter commented 8 years ago 
Documentation to be added to the docs section that talks about loading data

Original comment by vinay...@gmail.com on 17 May 2013 at 7:45