search

apache / arrow

Apache Arrow is a multi-language toolbox for accelerated data interchange and in-memory processing
https://arrow.apache.org/
Apache License 2.0
14.35k stars 3.49k forks source link

[Java] Exception in thread "main" org.apache.arrow.memory.OutOfMemoryException: Unable to allocate buffer of size 4 due to memory limit. Current allocation: 2147483646 #22592

Closed asfimport closed 5 years ago

asfimport commented 5 years ago

jdbc query results exceed native heap when using generous -Xmx settings.

for roughly 800 megabytes of csv/flatfile resultset, arrow is unable to house the contents in RAM long enough to persist to disk, without explicit knowledge beyond unit test sample code.

source: https://github.com/jnorthrup/jdbc2json/blob/master/src/main/java/com/fnreport/QueryToFeather.kt#L83


Exception in thread "main" org.apache.arrow.memory.OutOfMemoryException: Unable to allocate buffer of size 4 due to memory limit. Current allocation: 2147483646
        at org.apache.arrow.memory.BaseAllocator.buffer(BaseAllocator.java:307)
        at org.apache.arrow.memory.BaseAllocator.buffer(BaseAllocator.java:277)
        at org.apache.arrow.adapter.jdbc.JdbcToArrowUtils.updateVector(JdbcToArrowUtils.java:610)
        at org.apache.arrow.adapter.jdbc.JdbcToArrowUtils.jdbcToFieldVector(JdbcToArrowUtils.java:462)
        at org.apache.arrow.adapter.jdbc.JdbcToArrowUtils.jdbcToArrowVectors(JdbcToArrowUtils.java:396)
        at org.apache.arrow.adapter.jdbc.JdbcToArrow.sqlToArrow(JdbcToArrow.java:225)
        at org.apache.arrow.adapter.jdbc.JdbcToArrow.sqlToArrow(JdbcToArrow.java:187)
        at org.apache.arrow.adapter.jdbc.JdbcToArrow.sqlToArrow(JdbcToArrow.java:156)
        at com.fnreport.QueryToFeather$Companion.go(QueryToFeather.kt:83)
        at com.fnreport.QueryToFeather$Companion$main$1.invokeSuspend(QueryToFeather.kt:95)
        at kotlin.coroutines.jvm.internal.BaseContinuationImpl.resumeWith(ContinuationImpl.kt:33)
        at kotlinx.coroutines.DispatchedTask.run(Dispatched.kt:241)
        at kotlinx.coroutines.EventLoopImplBase.processNextEvent(EventLoop.common.kt:270)
        at kotlinx.coroutines.BlockingCoroutine.joinBlocking(Builders.kt:79)
        at kotlinx.coroutines.BuildersKt__BuildersKt.runBlocking(Builders.kt:54)
        at kotlinx.coroutines.BuildersKt.runBlocking(Unknown Source)
        at kotlinx.coroutines.BuildersKt__BuildersKt.runBlocking$default(Builders.kt:36)
        at kotlinx.coroutines.BuildersKt.runBlocking$default(Unknown Source)
        at com.fnreport.QueryToFeather$Companion.main(QueryToFeather.kt:93)
        at com.fnreport.QueryToFeather.main(QueryToFeather.kt)

Reporter: Jim Northrup Assignee: Micah Kornfield / @emkornfield

Related issues:

Note: This issue was originally created as ARROW-6202. Please see the migration documentation for further details.

asfimport commented 5 years ago

Micah Kornfield / @emkornfield: Is it possible to share a sample file or at least a schema.  A pretty substantial refactor was just checked in to master, do you have the ability to test against that?

asfimport commented 5 years ago

Micah Kornfield / @emkornfield: Specifically, I think there was at least one memory leak in the 0.14.1 code.

asfimport commented 5 years ago

Jim Northrup: full program execution below, stderr. record count 6.6 million records.


++ dirname ../../jdbc2json/bin/flatsql.sh
+ JDIR=../../jdbc2json/bin/../
+ exec java -classpath '../../jdbc2json/bin/..//target/*:../../jdbc2json/bin/..//target/lib/*' com.fnreport.QueryToFlat 'jdbc:informix-sqli://10.14.18.30:9088/i_l01:INFORMIXSERVER=****;user=****;password=****' 'select  b.so_invoice_date,
       sum(b.sol_shipped_qty) ag_qty,a.available_stock,
       b.item_code,
       b.so_whse_code,
       b.division,
       b.department,
       b.category,
       b.class,
       b.item_group,
       b.cust_group,
       b.cust_category,
       b.cust_segment,
       b.cust_subsegment,
       b.sol_price_rule,
       b.del_method
from informix.ZEMP_STOCK_DAILY as a
         inner join isnt.ZEMP_SALES_BY_ITEM_FOR_REPORT as b
                    on  
                            a.whse_code = b.so_whse_code
                            and a.stock_code = b.item_code
                            and a.stock_date = b.so_invoice_date
                            and b.isbulk = 0
group by a.available_stock,
         b.so_invoice_date,
         b.item_code,
         b.so_whse_code,
         b.division,
         b.department,
         b.category,
         b.class,
         b.item_group,
         b.cust_group,
         b.cust_category,
         b.cust_segment,
         b.cust_subsegment,
         b.sol_price_rule,
         b.del_method'
using sql: select  b.so_invoice_date,
       sum(b.sol_shipped_qty) ag_qty,a.available_stock,
       b.item_code,
       b.so_whse_code,
       b.division,
       b.department,
       b.category,
       b.class,
       b.item_group,
       b.cust_group,
       b.cust_category,
       b.cust_segment,
       b.cust_subsegment,
       b.sol_price_rule,
       b.del_method
from informix.ZEMP_STOCK_DAILY as a
         inner join isnt.ZEMP_SALES_BY_ITEM_FOR_REPORT as b
                    on  
                            a.whse_code = b.so_whse_code
                            and a.stock_code = b.item_code
                            and a.stock_date = b.so_invoice_date
                            and b.isbulk = 0
group by a.available_stock,
         b.so_invoice_date,
         b.item_code,
         b.so_whse_code,
         b.division,
         b.department,
         b.category,
         b.class,
         b.item_group,
         b.cust_group,
         b.cust_category,
         b.cust_segment,
         b.cust_subsegment,
         b.sol_price_rule,
         b.del_method
SLF4J: Failed to load class "org.slf4j.impl.StaticLoggerBinder".
SLF4J: Defaulting to no-operation (NOP) logger implementation
SLF4J: See http://www.slf4j.org/codes.html#StaticLoggerBinder for further details.
loading meta for ZEMP_STOCK_DAILY
(driver-specific potential meta, [TABLE_CAT, TABLE_SCHEM, TABLE_NAME, COLUMN_NAME, DATA_TYPE, TYPE_NAME, COLUMN_SIZE, BUFFER_LENGTH, DECIMAL_DIGITS, NUM_PREC_RADIX, NULLABLE, REMARKS, COLUMN_DEF, SQL_DATA_TYPE, SQL_DATETIME_SUB, CHAR_OCTET_LENGTH, ORDINAL_POSITION, IS_NULLABLE, SCOPE_CATALOG, SCOPE_SCHEMA, SCOPE_TABLE, SOURCE_DATA_TYPE, IS_AUTOINCREMENT])
loading meta for ZEMP_SALES_BY_ITEM_FOR_REPORT
(driver-specific potential meta, [TABLE_CAT, TABLE_SCHEM, TABLE_NAME, COLUMN_NAME, DATA_TYPE, TYPE_NAME, COLUMN_SIZE, BUFFER_LENGTH, DECIMAL_DIGITS, NUM_PREC_RADIX, NULLABLE, REMARKS, COLUMN_DEF, SQL_DATA_TYPE, SQL_DATETIME_SUB, CHAR_OCTET_LENGTH, ORDINAL_POSITION, IS_NULLABLE, SCOPE_CATALOG, SCOPE_SCHEMA, SCOPE_TABLE, SOURCE_DATA_TYPE, IS_AUTOINCREMENT])
[(ZEMP_STOCK_DAILY, [(whse_code, (DATE(13), 4)), (stock_date, (DOUBLE(7), 10)), (stock_code, (TINYINT(1), 16)), (stk_description, (TINYINT(1), 30)), (stk_apn_number, (TINYINT(1), 20)), (stock_group, (TINYINT(1), 4)), (sht_class, (TINYINT(1), 4)), (sht_category, (TINYINT(1), 4)), (sht_department, (TINYINT(1), 4)), (sht_division, (TINYINT(1), 4)), (stk_stock_status, (TINYINT(1), 1)), (whse_avg_cost, (FLOAT(5), 19)), (opening_balance, (FLOAT(5), 14)), (transfers_in, (FLOAT(5), 14)), (transfers_out, (FLOAT(5), 14)), (whse_adjustments, (FLOAT(5), 14)), (cm_sales_qty, (FLOAT(5), 14)), (on_consignment, (FLOAT(5), 14)), (whse_qty_on_hand, (FLOAT(5), 19)), (awaiting_putaway, (FLOAT(5), 14)), (factory_orders, (FLOAT(5), 14)), (whse_back_orders, (FLOAT(5), 14)), (forward_orders, (FLOAT(5), 14)), (current_orders, (FLOAT(5), 14)), (qty_on_order, (FLOAT(5), 14)), (qty_in_transit, (FLOAT(5), 14)), (available_stock, (FLOAT(5), 20)), (dms, (FLOAT(5), 14)), (dmspromo, (FLOAT(5), 14)), (buffer, (FLOAT(5), 14)), (pradms1, (FLOAT(5), 14)), (pradms2, (FLOAT(5), 14)), (time_insert, (CHAR(10), 23)), (lastbuydate, (DOUBLE(7), 10)), (kvi, (TINYINT(1), 4)), (condition_code, (TINYINT(1), 1)), (control_code, (TINYINT(1), 1)), (stock_last_sale, (DOUBLE(7), 10)), (consignment, (DATE(13), 2)), (receipt_date, (DOUBLE(7), 10)), (last_po_no, (SMALLINT(2), 10)), (last_po_backorder_flag, (DATE(13), 2)), (last_po_user_name, (DATE(13), 30)), (returnable_flag, (DATE(13), 1)), (calcdms, (FLOAT(5), 14)), (reorder_policy, (DATE(13), 1)), (seasonal, (DATE(13), 2)), (private_label, (DATE(13), 2)), (targetcusttype, (DATE(13), 3)), (tipe, (TINYINT(1), 10)), (creation_date, (DOUBLE(7), 10))]), (ZEMP_SALES_BY_ITEM_FOR_REPORT, [(so_whse_code, (DATE(13), 4)), (delivery, (DATE(13), 10)), (so_invoice_date, (DOUBLE(7), 10)), (so_invoice_day, (SMALLINT(2), 10)), (so_invoice_month, (SMALLINT(2), 10)), (so_invoice_year, (SMALLINT(2), 10)), (so_cust_code, (DATE(13), 10)), (so_order_no, (SMALLINT(2), 10)), (division, (DATE(13), 3)), (department, (DATE(13), 3)), (category, (DATE(13), 10)), (item_code, (DATE(13), 16)), (line_cost, (FLOAT(5), 18)), (shipped_ex_tax, (FLOAT(5), 32)), (shipped_amount, (FLOAT(5), 18)), (margin, (FLOAT(5), 18)), (margin_percentage, (FLOAT(5), 32)), (isbulk, (SMALLINT(2), 10)), (iscountedforinvoice, (SMALLINT(2), 10)), (class, (DATE(13), 4)), (item_group, (DATE(13), 4)), (item_description, (DATE(13), 30)), (so_order_status, (DATE(13), 2)), (sol_price_rule, (DATE(13), 2)), (sol_line_seq, (FLOAT(5), 7)), (sol_shipped_qty, (FLOAT(5), 28)), (so_whse_code_ori, (DATE(13), 4)), (cust_group, (DATE(13), 4)), (cust_category, (DATE(13), 4)), (cust_segment, (DATE(13), 4)), (cust_subsegment, (DATE(13), 4)), (del_method, (DATE(13), 10)), (member_type, (TINYINT(1), 2))])]
{"ZEMP_STOCK_DAILY":{"whse_code":{"TABLE_CAT":"i_l01","TABLE_SCHEM":"informix","TABLE_NAME":"zemp_stock_daily","COLUMN_NAME":"whse_code","DATA_TYPE":12,"TYPE_NAME":"varchar","COLUMN_SIZE":4,"BUFFER_LENGTH":0,"DECIMAL_DIGITS":0,"NUM_PREC_RADIX":0,"NULLABLE":1,"REMARKS":null,"COLUMN_DEF":null,"SQL_DATA_TYPE":13,"SQL_DATETIME_SUB":0,"CHAR_OCTET_LENGTH":4,"ORDINAL_POSITION":1,"IS_NULLABLE":"YES","SCOPE_CATALOG":null,"SCOPE_SCHEMA":null,"SCOPE_TABLE":null,"SOURCE_DATA_TYPE":null,"IS_AUTOINCREMENT":"NO"},"stock_date":{"TABLE_CAT":"i_l01","TABLE_SCHEM":"informix","TABLE_NAME":"zemp_stock_daily","COLUMN_NAME":"stock_date","DATA_TYPE":91,"TYPE_NAME":"date","COLUMN_SIZE":10,"BUFFER_LENGTH":0,"DECIMAL_DIGITS":0,"NUM_PREC_RADIX":0,"NULLABLE":1,"REMARKS":null,"COLUMN_DEF":null,"SQL_DATA_TYPE":7,"SQL_DATETIME_SUB":0,"CHAR_OCTET_LENGTH":0,"ORDINAL_POSITION":2,"IS_NULLABLE":"YES","SCOPE_CATALOG":null,"SCOPE_SCHEMA":null,"SCOPE_TABLE":null,"SOURCE_DATA_TYPE":null,"IS_AUTOINCREMENT":"NO"},"stock_code":{"TABLE_CAT":"i_l01","TABLE_SCHEM":"informix","TABLE_NAME":"zemp_stock_daily","COLUMN_NAME":"stock_code","DATA_TYPE":1,"TYPE_NAME":"char","COLUMN_SIZE":16,"BUFFER_LENGTH":0,"DECIMAL_DIGITS":0,"NUM_PREC_RADIX":0,"NULLABLE":1,"REMARKS":null,"COLUMN_DEF":null,"SQL_DATA_TYPE":0,"SQL_DATETIME_SUB":0,"CHAR_OCTET_LENGTH":16,"ORDINAL_POSITION":3,"IS_NULLABLE":"YES","SCOPE_CATALOG":null,"SCOPE_SCHEMA":null,"SCOPE_TABLE":null,"SOURCE_DATA_TYPE":null,"IS_AUTOINCREMENT":"NO"},"stk_description":{"TABLE_CAT":"i_l01","TABLE_SCHEM":"informix","TABLE_NAME":"zemp_stock_daily","COLUMN_NAME":"stk_description","DATA_TYPE":1,"TYPE_NAME":"char","COLUMN_SIZE":30,"BUFFER_LENGTH":0,"DECIMAL_DIGITS":0,"NUM_PREC_RADIX":0,"NULLABLE":1,"REMARKS":null,"COLUMN_DEF":null,"SQL_DATA_TYPE":0,"SQL_DATETIME_SUB":0,"CHAR_OCTET_LENGTH":30,"ORDINAL_POSITION":4,"IS_NULLABLE":"YES","SCOPE_CATALOG":null,"SCOPE_SCHEMA":null,"SCOPE_TABLE":null,"SOURCE_DATA_TYPE":null,"IS_AUTOINCREMENT":"NO"},"stk_apn_number":{"TABLE_CAT":"i_l01","TABLE_SCHEM":"informix","TABLE_NAME":"zemp_stock_daily","COLUMN_NAME":"stk_apn_number","DATA_TYPE":1,"TYPE_NAME":"char","COLUMN_SIZE":20,"BUFFER_LENGTH":0,"DECIMAL_DIGITS":0,"NUM_PREC_RADIX":0,"NULLABLE":1,"REMARKS":null,"COLUMN_DEF":null,"SQL_DATA_TYPE":0,"SQL_DATETIME_SUB":0,"CHAR_OCTET_LENGTH":20,"ORDINAL_POSITION":5,"IS_NULLABLE":"YES","SCOPE_CATALOG":null,"SCOPE_SCHEMA":null,"SCOPE_TABLE":null,"SOURCE_DATA_TYPE":null,"IS_AUTOINCREMENT":"NO"},"stock_group":{"TABLE_CAT":"i_l01","TABLE_SCHEM":"informix","TABLE_NAME":"zemp_stock_daily","COLUMN_NAME":"stock_group","DATA_TYPE":1,"TYPE_NAME":"char","COLUMN_SIZE":4,"BUFFER_LENGTH":0,"DECIMAL_DIGITS":0,"NUM_PREC_RADIX":0,"NULLABLE":1,"REMARKS":null,"COLUMN_DEF":null,"SQL_DATA_TYPE":0,"SQL_DATETIME_SUB":0,"CHAR_OCTET_LENGTH":4,"ORDINAL_POSITION":6,"IS_NULLABLE":"YES","SCOPE_CATALOG":null,"SCOPE_SCHEMA":null,"SCOPE_TABLE":null,"SOURCE_DATA_TYPE":null,"IS_AUTOINCREMENT":"NO"},"sht_class":{"TABLE_CAT":"i_l01","TABLE_SCHEM":"informix","TABLE_NAME":"zemp_stock_daily","COLUMN_NAME":"sht_class","DATA_TYPE":1,"TYPE_NAME":"char","COLUMN_SIZE":4,"BUFFER_LENGTH":0,"DECIMAL_DIGITS":0,"NUM_PREC_RADIX":0,"NULLABLE":1,"REMARKS":null,"COLUMN_DEF":null,"SQL_DATA_TYPE":0,"SQL_DATETIME_SUB":0,"CHAR_OCTET_LENGTH":4,"ORDINAL_POSITION":7,"IS_NULLABLE":"YES","SCOPE_CATALOG":null,"SCOPE_SCHEMA":null,"SCOPE_TABLE":null,"SOURCE_DATA_TYPE":null,"IS_AUTOINCREMENT":"NO"},"sht_category":{"TABLE_CAT":"i_l01","TABLE_SCHEM":"informix","TABLE_NAME":"zemp_stock_daily","COLUMN_NAME":"sht_category","DATA_TYPE":1,"TYPE_NAME":"char","COLUMN_SIZE":4,"BUFFER_LENGTH":0,"DECIMAL_DIGITS":0,"NUM_PREC_RADIX":0,"NULLABLE":1,"REMARKS":null,"COLUMN_DEF":null,"SQL_DATA_TYPE":0,"SQL_DATETIME_SUB":0,"CHAR_OCTET_LENGTH":4,"ORDINAL_POSITION":8,"IS_NULLABLE":"YES","SCOPE_CATALOG":null,"SCOPE_SCHEMA":null,"SCOPE_TABLE":null,"SOURCE_DATA_TYPE":null,"IS_AUTOINCREMENT":"NO"},"sht_department":{"TABLE_CAT":"i_l01","TABLE_SCHEM":"informix","TABLE_NAME":"zemp_stock_daily","COLUMN_NAME":"sht_department","DATA_TYPE":1,"TYPE_NAME":"char","COLUMN_SIZE":4,"BUFFER_LENGTH":0,"DECIMAL_DIGITS":0,"NUM_PREC_RADIX":0,"NULLABLE":1,"REMARKS":null,"COLUMN_DEF":null,"SQL_DATA_TYPE":0,"SQL_DATETIME_SUB":0,"CHAR_OCTET_LENGTH":4,"ORDINAL_POSITION":9,"IS_NULLABLE":"YES","SCOPE_CATALOG":null,"SCOPE_SCHEMA":null,"SCOPE_TABLE":null,"SOURCE_DATA_TYPE":null,"IS_AUTOINCREMENT":"NO"},"sht_division":{"TABLE_CAT":"i_l01","TABLE_SCHEM":"informix","TABLE_NAME":"zemp_stock_daily","COLUMN_NAME":"sht_division","DATA_TYPE":1,"TYPE_NAME":"char","COLUMN_SIZE":4,"BUFFER_LENGTH":0,"DECIMAL_DIGITS":0,"NUM_PREC_RADIX":0,"NULLABLE":1,"REMARKS":null,"COLUMN_DEF":null,"SQL_DATA_TYPE":0,"SQL_DATETIME_SUB":0,"CHAR_OCTET_LENGTH":4,"ORDINAL_POSITION":10,"IS_NULLABLE":"YES","SCOPE_CATALOG":null,"SCOPE_SCHEMA":null,"SCOPE_TABLE":null,"SOURCE_DATA_TYPE":null,"IS_AUTOINCREMENT":"NO"},"stk_stock_status":{"TABLE_CAT":"i_l01","TABLE_SCHEM":"informix","TABLE_NAME":"zemp_stock_daily","COLUMN_NAME":"stk_stock_status","DATA_TYPE":1,"TYPE_NAME":"char","COLUMN_SIZE":1,"BUFFER_LENGTH":0,"DECIMAL_DIGITS":0,"NUM_PREC_RADIX":0,"NULLABLE":1,"REMARKS":null,"COLUMN_DEF":null,"SQL_DATA_TYPE":0,"SQL_DATETIME_SUB":0,"CHAR_OCTET_LENGTH":1,"ORDINAL_POSITION":11,"IS_NULLABLE":"YES","SCOPE_CATALOG":null,"SCOPE_SCHEMA":null,"SCOPE_TABLE":null,"SOURCE_DATA_TYPE":null,"IS_AUTOINCREMENT":"NO"},"whse_avg_cost":{"TABLE_CAT":"i_l01","TABLE_SCHEM":"informix","TABLE_NAME":"zemp_stock_daily","COLUMN_NAME":"whse_avg_cost","DATA_TYPE":3,"TYPE_NAME":"decimal","COLUMN_SIZE":19,"BUFFER_LENGTH":0,"DECIMAL_DIGITS":4,"NUM_PREC_RADIX":10,"NULLABLE":1,"REMARKS":null,"COLUMN_DEF":null,"SQL_DATA_TYPE":5,"SQL_DATETIME_SUB":0,"CHAR_OCTET_LENGTH":0,"ORDINAL_POSITION":12,"IS_NULLABLE":"YES","SCOPE_CATALOG":null,"SCOPE_SCHEMA":null,"SCOPE_TABLE":null,"SOURCE_DATA_TYPE":null,"IS_AUTOINCREMENT":"NO"},"opening_balance":{"TABLE_CAT":"i_l01","TABLE_SCHEM":"informix","TABLE_NAME":"zemp_stock_daily","COLUMN_NAME":"opening_balance","DATA_TYPE":3,"TYPE_NAME":"decimal","COLUMN_SIZE":14,"BUFFER_LENGTH":0,"DECIMAL_DIGITS":4,"NUM_PREC_RADIX":10,"NULLABLE":1,"REMARKS":null,"COLUMN_DEF":null,"SQL_DATA_TYPE":5,"SQL_DATETIME_SUB":0,"CHAR_OCTET_LENGTH":0,"ORDINAL_POSITION":13,"IS_NULLABLE":"YES","SCOPE_CATALOG":null,"SCOPE_SCHEMA":null,"SCOPE_TABLE":null,"SOURCE_DATA_TYPE":null,"IS_AUTOINCREMENT":"NO"},"transfers_in":{"TABLE_CAT":"i_l01","TABLE_SCHEM":"informix","TABLE_NAME":"zemp_stock_daily","COLUMN_NAME":"transfers_in","DATA_TYPE":3,"TYPE_NAME":"decimal","COLUMN_SIZE":14,"BUFFER_LENGTH":0,"DECIMAL_DIGITS":4,"NUM_PREC_RADIX":10,"NULLABLE":1,"REMARKS":null,"COLUMN_DEF":null,"SQL_DATA_TYPE":5,"SQL_DATETIME_SUB":0,"CHAR_OCTET_LENGTH":0,"ORDINAL_POSITION":14,"IS_NULLABLE":"YES","SCOPE_CATALOG":null,"SCOPE_SCHEMA":null,"SCOPE_TABLE":null,"SOURCE_DATA_TYPE":null,"IS_AUTOINCREMENT":"NO"},"transfers_out":{"TABLE_CAT":"i_l01","TABLE_SCHEM":"informix","TABLE_NAME":"zemp_stock_daily","COLUMN_NAME":"transfers_out","DATA_TYPE":3,"TYPE_NAME":"decimal","COLUMN_SIZE":14,"BUFFER_LENGTH":0,"DECIMAL_DIGITS":4,"NUM_PREC_RADIX":10,"NULLABLE":1,"REMARKS":null,"COLUMN_DEF":null,"SQL_DATA_TYPE":5,"SQL_DATETIME_SUB":0,"CHAR_OCTET_LENGTH":0,"ORDINAL_POSITION":15,"IS_NULLABLE":"YES","SCOPE_CATALOG":null,"SCOPE_SCHEMA":null,"SCOPE_TABLE":null,"SOURCE_DATA_TYPE":null,"IS_AUTOINCREMENT":"NO"},"whse_adjustments":{"TABLE_CAT":"i_l01","TABLE_SCHEM":"informix","TABLE_NAME":"zemp_stock_daily","COLUMN_NAME":"whse_adjustments","DATA_TYPE":3,"TYPE_NAME":"decimal","COLUMN_SIZE":14,"BUFFER_LENGTH":0,"DECIMAL_DIGITS":4,"NUM_PREC_RADIX":10,"NULLABLE":1,"REMARKS":null,"COLUMN_DEF":null,"SQL_DATA_TYPE":5,"SQL_DATETIME_SUB":0,"CHAR_OCTET_LENGTH":0,"ORDINAL_POSITION":16,"IS_NULLABLE":"YES","SCOPE_CATALOG":null,"SCOPE_SCHEMA":null,"SCOPE_TABLE":null,"SOURCE_DATA_TYPE":null,"IS_AUTOINCREMENT":"NO"},"cm_sales_qty":{"TABLE_CAT":"i_l01","TABLE_SCHEM":"informix","TABLE_NAME":"zemp_stock_daily","COLUMN_NAME":"cm_sales_qty","DATA_TYPE":3,"TYPE_NAME":"decimal","COLUMN_SIZE":14,"BUFFER_LENGTH":0,"DECIMAL_DIGITS":4,"NUM_PREC_RADIX":10,"NULLABLE":1,"REMARKS":null,"COLUMN_DEF":null,"SQL_DATA_TYPE":5,"SQL_DATETIME_SUB":0,"CHAR_OCTET_LENGTH":0,"ORDINAL_POSITION":17,"IS_NULLABLE":"YES","SCOPE_CATALOG":null,"SCOPE_SCHEMA":null,"SCOPE_TABLE":null,"SOURCE_DATA_TYPE":null,"IS_AUTOINCREMENT":"NO"},"on_consignment":{"TABLE_CAT":"i_l01","TABLE_SCHEM":"informix","TABLE_NAME":"zemp_stock_daily","COLUMN_NAME":"on_consignment","DATA_TYPE":3,"TYPE_NAME":"decimal","COLUMN_SIZE":14,"BUFFER_LENGTH":0,"DECIMAL_DIGITS":4,"NUM_PREC_RADIX":10,"NULLABLE":1,"REMARKS":null,"COLUMN_DEF":null,"SQL_DATA_TYPE":5,"SQL_DATETIME_SUB":0,"CHAR_OCTET_LENGTH":0,"ORDINAL_POSITION":18,"IS_NULLABLE":"YES","SCOPE_CATALOG":null,"SCOPE_SCHEMA":null,"SCOPE_TABLE":null,"SOURCE_DATA_TYPE":null,"IS_AUTOINCREMENT":"NO"},"whse_qty_on_hand":{"TABLE_CAT":"i_l01","TABLE_SCHEM":"informix","TABLE_NAME":"zemp_stock_daily","COLUMN_NAME":"whse_qty_on_hand","DATA_TYPE":3,"TYPE_NAME":"decimal","COLUMN_SIZE":19,"BUFFER_LENGTH":0,"DECIMAL_DIGITS":4,"NUM_PREC_RADIX":10,"NULLABLE":1,"REMARKS":null,"COLUMN_DEF":null,"SQL_DATA_TYPE":5,"SQL_DATETIME_SUB":0,"CHAR_OCTET_LENGTH":0,"ORDINAL_POSITION":19,"IS_NULLABLE":"YES","SCOPE_CATALOG":null,"SCOPE_SCHEMA":null,"SCOPE_TABLE":null,"SOURCE_DATA_TYPE":null,"IS_AUTOINCREMENT":"NO"},"awaiting_putaway":{"TABLE_CAT":"i_l01","TABLE_SCHEM":"informix","TABLE_NAME":"zemp_stock_daily","COLUMN_NAME":"awaiting_putaway","DATA_TYPE":3,"TYPE_NAME":"decimal","COLUMN_SIZE":14,"BUFFER_LENGTH":0,"DECIMAL_DIGITS":4,"NUM_PREC_RADIX":10,"NULLABLE":1,"REMARKS":null,"COLUMN_DEF":null,"SQL_DATA_TYPE":5,"SQL_DATETIME_SUB":0,"CHAR_OCTET_LENGTH":0,"ORDINAL_POSITION":20,"IS_NULLABLE":"YES","SCOPE_CATALOG":null,"SCOPE_SCHEMA":null,"SCOPE_TABLE":null,"SOURCE_DATA_TYPE":null,"IS_AUTOINCREMENT":"NO"},"factory_orders":{"TABLE_CAT":"i_l01","TABLE_SCHEM":"informix","TABLE_NAME":"zemp_stock_daily","COLUMN_NAME":"factory_orders","DATA_TYPE":3,"TYPE_NAME":"decimal","COLUMN_SIZE":14,"BUFFER_LENGTH":0,"DECIMAL_DIGITS":4,"NUM_PREC_RADIX":10,"NULLABLE":1,"REMARKS":null,"COLUMN_DEF":null,"SQL_DATA_TYPE":5,"SQL_DATETIME_SUB":0,"CHAR_OCTET_LENGTH":0,"ORDINAL_POSITION":21,"IS_NULLABLE":"YES","SCOPE_CATALOG":null,"SCOPE_SCHEMA":null,"SCOPE_TABLE":null,"SOURCE_DATA_TYPE":null,"IS_AUTOINCREMENT":"NO"},"whse_back_orders":{"TABLE_CAT":"i_l01","TABLE_SCHEM":"informix","TABLE_NAME":"zemp_stock_daily","COLUMN_NAME":"whse_back_orders","DATA_TYPE":3,"TYPE_NAME":"decimal","COLUMN_SIZE":14,"BUFFER_LENGTH":0,"DECIMAL_DIGITS":4,"NUM_PREC_RADIX":10,"NULLABLE":1,"REMARKS":null,"COLUMN_DEF":null,"SQL_DATA_TYPE":5,"SQL_DATETIME_SUB":0,"CHAR_OCTET_LENGTH":0,"ORDINAL_POSITION":22,"IS_NULLABLE":"YES","SCOPE_CATALOG":null,"SCOPE_SCHEMA":null,"SCOPE_TABLE":null,"SOURCE_DATA_TYPE":null,"IS_AUTOINCREMENT":"NO"},"forward_orders":{"TABLE_CAT":"i_l01","TABLE_SCHEM":"informix","TABLE_NAME":"zemp_stock_daily","COLUMN_NAME":"forward_orders","DATA_TYPE":3,"TYPE_NAME":"decimal","COLUMN_SIZE":14,"BUFFER_LENGTH":0,"DECIMAL_DIGITS":4,"NUM_PREC_RADIX":10,"NULLABLE":1,"REMARKS":null,"COLUMN_DEF":null,"SQL_DATA_TYPE":5,"SQL_DATETIME_SUB":0,"CHAR_OCTET_LENGTH":0,"ORDINAL_POSITION":23,"IS_NULLABLE":"YES","SCOPE_CATALOG":null,"SCOPE_SCHEMA":null,"SCOPE_TABLE":null,"SOURCE_DATA_TYPE":null,"IS_AUTOINCREMENT":"NO"},"current_orders":{"TABLE_CAT":"i_l01","TABLE_SCHEM":"informix","TABLE_NAME":"zemp_stock_daily","COLUMN_NAME":"current_orders","DATA_TYPE":3,"TYPE_NAME":"decimal","COLUMN_SIZE":14,"BUFFER_LENGTH":0,"DECIMAL_DIGITS":4,"NUM_PREC_RADIX":10,"NULLABLE":1,"REMARKS":null,"COLUMN_DEF":null,"SQL_DATA_TYPE":5,"SQL_DATETIME_SUB":0,"CHAR_OCTET_LENGTH":0,"ORDINAL_POSITION":24,"IS_NULLABLE":"YES","SCOPE_CATALOG":null,"SCOPE_SCHEMA":null,"SCOPE_TABLE":null,"SOURCE_DATA_TYPE":null,"IS_AUTOINCREMENT":"NO"},"qty_on_order":{"TABLE_CAT":"i_l01","TABLE_SCHEM":"informix","TABLE_NAME":"zemp_stock_daily","COLUMN_NAME":"qty_on_order","DATA_TYPE":3,"TYPE_NAME":"decimal","COLUMN_SIZE":14,"BUFFER_LENGTH":0,"DECIMAL_DIGITS":4,"NUM_PREC_RADIX":10,"NULLABLE":1,"REMARKS":null,"COLUMN_DEF":null,"SQL_DATA_TYPE":5,"SQL_DATETIME_SUB":0,"CHAR_OCTET_LENGTH":0,"ORDINAL_POSITION":25,"IS_NULLABLE":"YES","SCOPE_CATALOG":null,"SCOPE_SCHEMA":null,"SCOPE_TABLE":null,"SOURCE_DATA_TYPE":null,"IS_AUTOINCREMENT":"NO"},"qty_in_transit":{"TABLE_CAT":"i_l01","TABLE_SCHEM":"informix","TABLE_NAME":"zemp_stock_daily","COLUMN_NAME":"qty_in_transit","DATA_TYPE":3,"TYPE_NAME":"decimal","COLUMN_SIZE":14,"BUFFER_LENGTH":0,"DECIMAL_DIGITS":4,"NUM_PREC_RADIX":10,"NULLABLE":1,"REMARKS":null,"COLUMN_DEF":null,"SQL_DATA_TYPE":5,"SQL_DATETIME_SUB":0,"CHAR_OCTET_LENGTH":0,"ORDINAL_POSITION":26,"IS_NULLABLE":"YES","SCOPE_CATALOG":null,"SCOPE_SCHEMA":null,"SCOPE_TABLE":null,"SOURCE_DATA_TYPE":null,"IS_AUTOINCREMENT":"NO"},"available_stock":{"TABLE_CAT":"i_l01","TABLE_SCHEM":"informix","TABLE_NAME":"zemp_stock_daily","COLUMN_NAME":"available_stock","DATA_TYPE":3,"TYPE_NAME":"decimal","COLUMN_SIZE":20,"BUFFER_LENGTH":0,"DECIMAL_DIGITS":4,"NUM_PREC_RADIX":10,"NULLABLE":1,"REMARKS":null,"COLUMN_DEF":null,"SQL_DATA_TYPE":5,"SQL_DATETIME_SUB":0,"CHAR_OCTET_LENGTH":0,"ORDINAL_POSITION":27,"IS_NULLABLE":"YES","SCOPE_CATALOG":null,"SCOPE_SCHEMA":null,"SCOPE_TABLE":null,"SOURCE_DATA_TYPE":null,"IS_AUTOINCREMENT":"NO"},"dms":{"TABLE_CAT":"i_l01","TABLE_SCHEM":"informix","TABLE_NAME":"zemp_stock_daily","COLUMN_NAME":"dms","DATA_TYPE":3,"TYPE_NAME":"decimal","COLUMN_SIZE":14,"BUFFER_LENGTH":0,"DECIMAL_DIGITS":4,"NUM_PREC_RADIX":10,"NULLABLE":1,"REMARKS":null,"COLUMN_DEF":null,"SQL_DATA_TYPE":5,"SQL_DATETIME_SUB":0,"CHAR_OCTET_LENGTH":0,"ORDINAL_POSITION":28,"IS_NULLABLE":"YES","SCOPE_CATALOG":null,"SCOPE_SCHEMA":null,"SCOPE_TABLE":null,"SOURCE_DATA_TYPE":null,"IS_AUTOINCREMENT":"NO"},"dmspromo":{"TABLE_CAT":"i_l01","TABLE_SCHEM":"informix","TABLE_NAME":"zemp_stock_daily","COLUMN_NAME":"dmspromo","DATA_TYPE":3,"TYPE_NAME":"decimal","COLUMN_SIZE":14,"BUFFER_LENGTH":0,"DECIMAL_DIGITS":4,"NUM_PREC_RADIX":10,"NULLABLE":1,"REMARKS":null,"COLUMN_DEF":null,"SQL_DATA_TYPE":5,"SQL_DATETIME_SUB":0,"CHAR_OCTET_LENGTH":0,"ORDINAL_POSITION":29,"IS_NULLABLE":"YES","SCOPE_CATALOG":null,"SCOPE_SCHEMA":null,"SCOPE_TABLE":null,"SOURCE_DATA_TYPE":null,"IS_AUTOINCREMENT":"NO"},"buffer":{"TABLE_CAT":"i_l01","TABLE_SCHEM":"informix","TABLE_NAME":"zemp_stock_daily","COLUMN_NAME":"buffer","DATA_TYPE":3,"TYPE_NAME":"decimal","COLUMN_SIZE":14,"BUFFER_LENGTH":0,"DECIMAL_DIGITS":4,"NUM_PREC_RADIX":10,"NULLABLE":1,"REMARKS":null,"COLUMN_DEF":null,"SQL_DATA_TYPE":5,"SQL_DATETIME_SUB":0,"CHAR_OCTET_LENGTH":0,"ORDINAL_POSITION":30,"IS_NULLABLE":"YES","SCOPE_CATALOG":null,"SCOPE_SCHEMA":null,"SCOPE_TABLE":null,"SOURCE_DATA_TYPE":null,"IS_AUTOINCREMENT":"NO"},"pradms1":{"TABLE_CAT":"i_l01","TABLE_SCHEM":"informix","TABLE_NAME":"zemp_stock_daily","COLUMN_NAME":"pradms1","DATA_TYPE":3,"TYPE_NAME":"decimal","COLUMN_SIZE":14,"BUFFER_LENGTH":0,"DECIMAL_DIGITS":4,"NUM_PREC_RADIX":10,"NULLABLE":1,"REMARKS":null,"COLUMN_DEF":null,"SQL_DATA_TYPE":5,"SQL_DATETIME_SUB":0,"CHAR_OCTET_LENGTH":0,"ORDINAL_POSITION":31,"IS_NULLABLE":"YES","SCOPE_CATALOG":null,"SCOPE_SCHEMA":null,"SCOPE_TABLE":null,"SOURCE_DATA_TYPE":null,"IS_AUTOINCREMENT":"NO"},"pradms2":{"TABLE_CAT":"i_l01","TABLE_SCHEM":"informix","TABLE_NAME":"zemp_stock_daily","COLUMN_NAME":"pradms2","DATA_TYPE":3,"TYPE_NAME":"decimal","COLUMN_SIZE":14,"BUFFER_LENGTH":0,"DECIMAL_DIGITS":4,"NUM_PREC_RADIX":10,"NULLABLE":1,"REMARKS":null,"COLUMN_DEF":null,"SQL_DATA_TYPE":5,"SQL_DATETIME_SUB":0,"CHAR_OCTET_LENGTH":0,"ORDINAL_POSITION":32,"IS_NULLABLE":"YES","SCOPE_CATALOG":null,"SCOPE_SCHEMA":null,"SCOPE_TABLE":null,"SOURCE_DATA_TYPE":null,"IS_AUTOINCREMENT":"NO"},"time_insert":{"TABLE_CAT":"i_l01","TABLE_SCHEM":"informix","TABLE_NAME":"zemp_stock_daily","COLUMN_NAME":"time_insert","DATA_TYPE":93,"TYPE_NAME":"datetime year to fraction(3)","COLUMN_SIZE":23,"BUFFER_LENGTH":0,"DECIMAL_DIGITS":0,"NUM_PREC_RADIX":0,"NULLABLE":1,"REMARKS":null,"COLUMN_DEF":null,"SQL_DATA_TYPE":10,"SQL_DATETIME_SUB":0,"CHAR_OCTET_LENGTH":0,"ORDINAL_POSITION":33,"IS_NULLABLE":"YES","SCOPE_CATALOG":null,"SCOPE_SCHEMA":null,"SCOPE_TABLE":null,"SOURCE_DATA_TYPE":null,"IS_AUTOINCREMENT":"NO"},"lastbuydate":{"TABLE_CAT":"i_l01","TABLE_SCHEM":"informix","TABLE_NAME":"zemp_stock_daily","COLUMN_NAME":"lastbuydate","DATA_TYPE":91,"TYPE_NAME":"date","COLUMN_SIZE":10,"BUFFER_LENGTH":0,"DECIMAL_DIGITS":0,"NUM_PREC_RADIX":0,"NULLABLE":1,"REMARKS":null,"COLUMN_DEF":null,"SQL_DATA_TYPE":7,"SQL_DATETIME_SUB":0,"CHAR_OCTET_LENGTH":0,"ORDINAL_POSITION":34,"IS_NULLABLE":"YES","SCOPE_CATALOG":null,"SCOPE_SCHEMA":null,"SCOPE_TABLE":null,"SOURCE_DATA_TYPE":null,"IS_AUTOINCREMENT":"NO"},"kvi":{"TABLE_CAT":"i_l01","TABLE_SCHEM":"informix","TABLE_NAME":"zemp_stock_daily","COLUMN_NAME":"kvi","DATA_TYPE":1,"TYPE_NAME":"char","COLUMN_SIZE":4,"BUFFER_LENGTH":0,"DECIMAL_DIGITS":0,"NUM_PREC_RADIX":0,"NULLABLE":1,"REMARKS":null,"COLUMN_DEF":null,"SQL_DATA_TYPE":0,"SQL_DATETIME_SUB":0,"CHAR_OCTET_LENGTH":4,"ORDINAL_POSITION":35,"IS_NULLABLE":"YES","SCOPE_CATALOG":null,"SCOPE_SCHEMA":null,"SCOPE_TABLE":null,"SOURCE_DATA_TYPE":null,"IS_AUTOINCREMENT":"NO"},"condition_code":{"TABLE_CAT":"i_l01","TABLE_SCHEM":"informix","TABLE_NAME":"zemp_stock_daily","COLUMN_NAME":"condition_code","DATA_TYPE":1,"TYPE_NAME":"char","COLUMN_SIZE":1,"BUFFER_LENGTH":0,"DECIMAL_DIGITS":0,"NUM_PREC_RADIX":0,"NULLABLE":1,"REMARKS":null,"COLUMN_DEF":null,"SQL_DATA_TYPE":0,"SQL_DATETIME_SUB":0,"CHAR_OCTET_LENGTH":1,"ORDINAL_POSITION":36,"IS_NULLABLE":"YES","SCOPE_CATALOG":null,"SCOPE_SCHEMA":null,"SCOPE_TABLE":null,"SOURCE_DATA_TYPE":null,"IS_AUTOINCREMENT":"NO"},"control_code":{"TABLE_CAT":"i_l01","TABLE_SCHEM":"informix","TABLE_NAME":"zemp_stock_daily","COLUMN_NAME":"control_code","DATA_TYPE":1,"TYPE_NAME":"char","COLUMN_SIZE":1,"BUFFER_LENGTH":0,"DECIMAL_DIGITS":0,"NUM_PREC_RADIX":0,"NULLABLE":1,"REMARKS":null,"COLUMN_DEF":null,"SQL_DATA_TYPE":0,"SQL_DATETIME_SUB":0,"CHAR_OCTET_LENGTH":1,"ORDINAL_POSITION":37,"IS_NULLABLE":"YES","SCOPE_CATALOG":null,"SCOPE_SCHEMA":null,"SCOPE_TABLE":null,"SOURCE_DATA_TYPE":null,"IS_AUTOINCREMENT":"NO"},"stock_last_sale":{"TABLE_CAT":"i_l01","TABLE_SCHEM":"informix","TABLE_NAME":"zemp_stock_daily","COLUMN_NAME":"stock_last_sale","DATA_TYPE":91,"TYPE_NAME":"date","COLUMN_SIZE":10,"BUFFER_LENGTH":0,"DECIMAL_DIGITS":0,"NUM_PREC_RADIX":0,"NULLABLE":1,"REMARKS":null,"COLUMN_DEF":null,"SQL_DATA_TYPE":7,"SQL_DATETIME_SUB":0,"CHAR_OCTET_LENGTH":0,"ORDINAL_POSITION":38,"IS_NULLABLE":"YES","SCOPE_CATALOG":null,"SCOPE_SCHEMA":null,"SCOPE_TABLE":null,"SOURCE_DATA_TYPE":null,"IS_AUTOINCREMENT":"NO"},"consignment":{"TABLE_CAT":"i_l01","TABLE_SCHEM":"informix","TABLE_NAME":"zemp_stock_daily","COLUMN_NAME":"consignment","DATA_TYPE":12,"TYPE_NAME":"varchar","COLUMN_SIZE":2,"BUFFER_LENGTH":0,"DECIMAL_DIGITS":0,"NUM_PREC_RADIX":0,"NULLABLE":1,"REMARKS":null,"COLUMN_DEF":null,"SQL_DATA_TYPE":13,"SQL_DATETIME_SUB":0,"CHAR_OCTET_LENGTH":2,"ORDINAL_POSITION":39,"IS_NULLABLE":"YES","SCOPE_CATALOG":null,"SCOPE_SCHEMA":null,"SCOPE_TABLE":null,"SOURCE_DATA_TYPE":null,"IS_AUTOINCREMENT":"NO"},"receipt_date":{"TABLE_CAT":"i_l01","TABLE_SCHEM":"informix","TABLE_NAME":"zemp_stock_daily","COLUMN_NAME":"receipt_date","DATA_TYPE":91,"TYPE_NAME":"date","COLUMN_SIZE":10,"BUFFER_LENGTH":0,"DECIMAL_DIGITS":0,"NUM_PREC_RADIX":0,"NULLABLE":1,"REMARKS":null,"COLUMN_DEF":null,"SQL_DATA_TYPE":7,"SQL_DATETIME_SUB":0,"CHAR_OCTET_LENGTH":0,"ORDINAL_POSITION":40,"IS_NULLABLE":"YES","SCOPE_CATALOG":null,"SCOPE_SCHEMA":null,"SCOPE_TABLE":null,"SOURCE_DATA_TYPE":null,"IS_AUTOINCREMENT":"NO"},"last_po_no":{"TABLE_CAT":"i_l01","TABLE_SCHEM":"informix","TABLE_NAME":"zemp_stock_daily","COLUMN_NAME":"last_po_no","DATA_TYPE":4,"TYPE_NAME":"integer","COLUMN_SIZE":10,"BUFFER_LENGTH":0,"DECIMAL_DIGITS":0,"NUM_PREC_RADIX":10,"NULLABLE":1,"REMARKS":null,"COLUMN_DEF":null,"SQL_DATA_TYPE":2,"SQL_DATETIME_SUB":0,"CHAR_OCTET_LENGTH":0,"ORDINAL_POSITION":41,"IS_NULLABLE":"YES","SCOPE_CATALOG":null,"SCOPE_SCHEMA":null,"SCOPE_TABLE":null,"SOURCE_DATA_TYPE":null,"IS_AUTOINCREMENT":"NO"},"last_po_backorder_flag":{"TABLE_CAT":"i_l01","TABLE_SCHEM":"informix","TABLE_NAME":"zemp_stock_daily","COLUMN_NAME":"last_po_backorder_flag","DATA_TYPE":12,"TYPE_NAME":"varchar","COLUMN_SIZE":2,"BUFFER_LENGTH":0,"DECIMAL_DIGITS":0,"NUM_PREC_RADIX":0,"NULLABLE":1,"REMARKS":null,"COLUMN_DEF":null,"SQL_DATA_TYPE":13,"SQL_DATETIME_SUB":0,"CHAR_OCTET_LENGTH":2,"ORDINAL_POSITION":42,"IS_NULLABLE":"YES","SCOPE_CATALOG":null,"SCOPE_SCHEMA":null,"SCOPE_TABLE":null,"SOURCE_DATA_TYPE":null,"IS_AUTOINCREMENT":"NO"},"last_po_user_name":{"TABLE_CAT":"i_l01","TABLE_SCHEM":"informix","TABLE_NAME":"zemp_stock_daily","COLUMN_NAME":"last_po_user_name","DATA_TYPE":12,"TYPE_NAME":"varchar","COLUMN_SIZE":30,"BUFFER_LENGTH":0,"DECIMAL_DIGITS":0,"NUM_PREC_RADIX":0,"NULLABLE":1,"REMARKS":null,"COLUMN_DEF":null,"SQL_DATA_TYPE":13,"SQL_DATETIME_SUB":0,"CHAR_OCTET_LENGTH":30,"ORDINAL_POSITION":43,"IS_NULLABLE":"YES","SCOPE_CATALOG":null,"SCOPE_SCHEMA":null,"SCOPE_TABLE":null,"SOURCE_DATA_TYPE":null,"IS_AUTOINCREMENT":"NO"},"returnable_flag":{"TABLE_CAT":"i_l01","TABLE_SCHEM":"informix","TABLE_NAME":"zemp_stock_daily","COLUMN_NAME":"returnable_flag","DATA_TYPE":12,"TYPE_NAME":"varchar","COLUMN_SIZE":1,"BUFFER_LENGTH":0,"DECIMAL_DIGITS":0,"NUM_PREC_RADIX":0,"NULLABLE":1,"REMARKS":null,"COLUMN_DEF":null,"SQL_DATA_TYPE":13,"SQL_DATETIME_SUB":0,"CHAR_OCTET_LENGTH":1,"ORDINAL_POSITION":44,"IS_NULLABLE":"YES","SCOPE_CATALOG":null,"SCOPE_SCHEMA":null,"SCOPE_TABLE":null,"SOURCE_DATA_TYPE":null,"IS_AUTOINCREMENT":"NO"},"calcdms":{"TABLE_CAT":"i_l01","TABLE_SCHEM":"informix","TABLE_NAME":"zemp_stock_daily","COLUMN_NAME":"calcdms","DATA_TYPE":3,"TYPE_NAME":"decimal","COLUMN_SIZE":14,"BUFFER_LENGTH":0,"DECIMAL_DIGITS":4,"NUM_PREC_RADIX":10,"NULLABLE":1,"REMARKS":null,"COLUMN_DEF":null,"SQL_DATA_TYPE":5,"SQL_DATETIME_SUB":0,"CHAR_OCTET_LENGTH":0,"ORDINAL_POSITION":45,"IS_NULLABLE":"YES","SCOPE_CATALOG":null,"SCOPE_SCHEMA":null,"SCOPE_TABLE":null,"SOURCE_DATA_TYPE":null,"IS_AUTOINCREMENT":"NO"},"reorder_policy":{"TABLE_CAT":"i_l01","TABLE_SCHEM":"informix","TABLE_NAME":"zemp_stock_daily","COLUMN_NAME":"reorder_policy","DATA_TYPE":12,"TYPE_NAME":"varchar","COLUMN_SIZE":1,"BUFFER_LENGTH":0,"DECIMAL_DIGITS":0,"NUM_PREC_RADIX":0,"NULLABLE":1,"REMARKS":null,"COLUMN_DEF":null,"SQL_DATA_TYPE":13,"SQL_DATETIME_SUB":0,"CHAR_OCTET_LENGTH":1,"ORDINAL_POSITION":46,"IS_NULLABLE":"YES","SCOPE_CATALOG":null,"SCOPE_SCHEMA":null,"SCOPE_TABLE":null,"SOURCE_DATA_TYPE":null,"IS_AUTOINCREMENT":"NO"},"seasonal":{"TABLE_CAT":"i_l01","TABLE_SCHEM":"informix","TABLE_NAME":"zemp_stock_daily","COLUMN_NAME":"seasonal","DATA_TYPE":12,"TYPE_NAME":"varchar","COLUMN_SIZE":2,"BUFFER_LENGTH":0,"DECIMAL_DIGITS":0,"NUM_PREC_RADIX":0,"NULLABLE":1,"REMARKS":null,"COLUMN_DEF":null,"SQL_DATA_TYPE":13,"SQL_DATETIME_SUB":0,"CHAR_OCTET_LENGTH":2,"ORDINAL_POSITION":47,"IS_NULLABLE":"YES","SCOPE_CATALOG":null,"SCOPE_SCHEMA":null,"SCOPE_TABLE":null,"SOURCE_DATA_TYPE":null,"IS_AUTOINCREMENT":"NO"},"private_label":{"TABLE_CAT":"i_l01","TABLE_SCHEM":"informix","TABLE_NAME":"zemp_stock_daily","COLUMN_NAME":"private_label","DATA_TYPE":12,"TYPE_NAME":"varchar","COLUMN_SIZE":2,"BUFFER_LENGTH":0,"DECIMAL_DIGITS":0,"NUM_PREC_RADIX":0,"NULLABLE":1,"REMARKS":null,"COLUMN_DEF":null,"SQL_DATA_TYPE":13,"SQL_DATETIME_SUB":0,"CHAR_OCTET_LENGTH":2,"ORDINAL_POSITION":48,"IS_NULLABLE":"YES","SCOPE_CATALOG":null,"SCOPE_SCHEMA":null,"SCOPE_TABLE":null,"SOURCE_DATA_TYPE":null,"IS_AUTOINCREMENT":"NO"},"targetcusttype":{"TABLE_CAT":"i_l01","TABLE_SCHEM":"informix","TABLE_NAME":"zemp_stock_daily","COLUMN_NAME":"targetcusttype","DATA_TYPE":12,"TYPE_NAME":"varchar","COLUMN_SIZE":3,"BUFFER_LENGTH":0,"DECIMAL_DIGITS":0,"NUM_PREC_RADIX":0,"NULLABLE":1,"REMARKS":null,"COLUMN_DEF":null,"SQL_DATA_TYPE":13,"SQL_DATETIME_SUB":0,"CHAR_OCTET_LENGTH":3,"ORDINAL_POSITION":49,"IS_NULLABLE":"YES","SCOPE_CATALOG":null,"SCOPE_SCHEMA":null,"SCOPE_TABLE":null,"SOURCE_DATA_TYPE":null,"IS_AUTOINCREMENT":"NO"},"tipe":{"TABLE_CAT":"i_l01","TABLE_SCHEM":"informix","TABLE_NAME":"zemp_stock_daily","COLUMN_NAME":"tipe","DATA_TYPE":1,"TYPE_NAME":"char","COLUMN_SIZE":10,"BUFFER_LENGTH":0,"DECIMAL_DIGITS":0,"NUM_PREC_RADIX":0,"NULLABLE":1,"REMARKS":null,"COLUMN_DEF":null,"SQL_DATA_TYPE":0,"SQL_DATETIME_SUB":0,"CHAR_OCTET_LENGTH":10,"ORDINAL_POSITION":50,"IS_NULLABLE":"YES","SCOPE_CATALOG":null,"SCOPE_SCHEMA":null,"SCOPE_TABLE":null,"SOURCE_DATA_TYPE":null,"IS_AUTOINCREMENT":"NO"},"creation_date":{"TABLE_CAT":"i_l01","TABLE_SCHEM":"informix","TABLE_NAME":"zemp_stock_daily","COLUMN_NAME":"creation_date","DATA_TYPE":91,"TYPE_NAME":"date","COLUMN_SIZE":10,"BUFFER_LENGTH":0,"DECIMAL_DIGITS":0,"NUM_PREC_RADIX":0,"NULLABLE":1,"REMARKS":null,"COLUMN_DEF":null,"SQL_DATA_TYPE":7,"SQL_DATETIME_SUB":0,"CHAR_OCTET_LENGTH":0,"ORDINAL_POSITION":51,"IS_NULLABLE":"YES","SCOPE_CATALOG":null,"SCOPE_SCHEMA":null,"SCOPE_TABLE":null,"SOURCE_DATA_TYPE":null,"IS_AUTOINCREMENT":"NO"}},"ZEMP_SALES_BY_ITEM_FOR_REPORT":{"so_whse_code":{"TABLE_CAT":"i_l01","TABLE_SCHEM":"isnt","TABLE_NAME":"zemp_sales_by_item_for_report","COLUMN_NAME":"so_whse_code","DATA_TYPE":12,"TYPE_NAME":"varchar","COLUMN_SIZE":4,"BUFFER_LENGTH":0,"DECIMAL_DIGITS":0,"NUM_PREC_RADIX":0,"NULLABLE":1,"REMARKS":null,"COLUMN_DEF":null,"SQL_DATA_TYPE":13,"SQL_DATETIME_SUB":0,"CHAR_OCTET_LENGTH":4,"ORDINAL_POSITION":1,"IS_NULLABLE":"YES","SCOPE_CATALOG":null,"SCOPE_SCHEMA":null,"SCOPE_TABLE":null,"SOURCE_DATA_TYPE":null,"IS_AUTOINCREMENT":"NO"},"delivery":{"TABLE_CAT":"i_l01","TABLE_SCHEM":"isnt","TABLE_NAME":"zemp_sales_by_item_for_report","COLUMN_NAME":"delivery","DATA_TYPE":12,"TYPE_NAME":"varchar","COLUMN_SIZE":10,"BUFFER_LENGTH":0,"DECIMAL_DIGITS":0,"NUM_PREC_RADIX":0,"NULLABLE":1,"REMARKS":null,"COLUMN_DEF":null,"SQL_DATA_TYPE":13,"SQL_DATETIME_SUB":0,"CHAR_OCTET_LENGTH":10,"ORDINAL_POSITION":2,"IS_NULLABLE":"YES","SCOPE_CATALOG":null,"SCOPE_SCHEMA":null,"SCOPE_TABLE":null,"SOURCE_DATA_TYPE":null,"IS_AUTOINCREMENT":"NO"},"so_invoice_date":{"TABLE_CAT":"i_l01","TABLE_SCHEM":"isnt","TABLE_NAME":"zemp_sales_by_item_for_report","COLUMN_NAME":"so_invoice_date","DATA_TYPE":91,"TYPE_NAME":"date","COLUMN_SIZE":10,"BUFFER_LENGTH":0,"DECIMAL_DIGITS":0,"NUM_PREC_RADIX":0,"NULLABLE":1,"REMARKS":null,"COLUMN_DEF":null,"SQL_DATA_TYPE":7,"SQL_DATETIME_SUB":0,"CHAR_OCTET_LENGTH":0,"ORDINAL_POSITION":3,"IS_NULLABLE":"YES","SCOPE_CATALOG":null,"SCOPE_SCHEMA":null,"SCOPE_TABLE":null,"SOURCE_DATA_TYPE":null,"IS_AUTOINCREMENT":"NO"},"so_invoice_day":{"TABLE_CAT":"i_l01","TABLE_SCHEM":"isnt","TABLE_NAME":"zemp_sales_by_item_for_report","COLUMN_NAME":"so_invoice_day","DATA_TYPE":4,"TYPE_NAME":"integer","COLUMN_SIZE":10,"BUFFER_LENGTH":0,"DECIMAL_DIGITS":0,"NUM_PREC_RADIX":10,"NULLABLE":1,"REMARKS":null,"COLUMN_DEF":null,"SQL_DATA_TYPE":2,"SQL_DATETIME_SUB":0,"CHAR_OCTET_LENGTH":0,"ORDINAL_POSITION":4,"IS_NULLABLE":"YES","SCOPE_CATALOG":null,"SCOPE_SCHEMA":null,"SCOPE_TABLE":null,"SOURCE_DATA_TYPE":null,"IS_AUTOINCREMENT":"NO"},"so_invoice_month":{"TABLE_CAT":"i_l01","TABLE_SCHEM":"isnt","TABLE_NAME":"zemp_sales_by_item_for_report","COLUMN_NAME":"so_invoice_month","DATA_TYPE":4,"TYPE_NAME":"integer","COLUMN_SIZE":10,"BUFFER_LENGTH":0,"DECIMAL_DIGITS":0,"NUM_PREC_RADIX":10,"NULLABLE":1,"REMARKS":null,"COLUMN_DEF":null,"SQL_DATA_TYPE":2,"SQL_DATETIME_SUB":0,"CHAR_OCTET_LENGTH":0,"ORDINAL_POSITION":5,"IS_NULLABLE":"YES","SCOPE_CATALOG":null,"SCOPE_SCHEMA":null,"SCOPE_TABLE":null,"SOURCE_DATA_TYPE":null,"IS_AUTOINCREMENT":"NO"},"so_invoice_year":{"TABLE_CAT":"i_l01","TABLE_SCHEM":"isnt","TABLE_NAME":"zemp_sales_by_item_for_report","COLUMN_NAME":"so_invoice_year","DATA_TYPE":4,"TYPE_NAME":"integer","COLUMN_SIZE":10,"BUFFER_LENGTH":0,"DECIMAL_DIGITS":0,"NUM_PREC_RADIX":10,"NULLABLE":1,"REMARKS":null,"COLUMN_DEF":null,"SQL_DATA_TYPE":2,"SQL_DATETIME_SUB":0,"CHAR_OCTET_LENGTH":0,"ORDINAL_POSITION":6,"IS_NULLABLE":"YES","SCOPE_CATALOG":null,"SCOPE_SCHEMA":null,"SCOPE_TABLE":null,"SOURCE_DATA_TYPE":null,"IS_AUTOINCREMENT":"NO"},"so_cust_code":{"TABLE_CAT":"i_l01","TABLE_SCHEM":"isnt","TABLE_NAME":"zemp_sales_by_item_for_report","COLUMN_NAME":"so_cust_code","DATA_TYPE":12,"TYPE_NAME":"varchar","COLUMN_SIZE":10,"BUFFER_LENGTH":0,"DECIMAL_DIGITS":0,"NUM_PREC_RADIX":0,"NULLABLE":1,"REMARKS":null,"COLUMN_DEF":null,"SQL_DATA_TYPE":13,"SQL_DATETIME_SUB":0,"CHAR_OCTET_LENGTH":10,"ORDINAL_POSITION":7,"IS_NULLABLE":"YES","SCOPE_CATALOG":null,"SCOPE_SCHEMA":null,"SCOPE_TABLE":null,"SOURCE_DATA_TYPE":null,"IS_AUTOINCREMENT":"NO"},"so_order_no":{"TABLE_CAT":"i_l01","TABLE_SCHEM":"isnt","TABLE_NAME":"zemp_sales_by_item_for_report","COLUMN_NAME":"so_order_no","DATA_TYPE":4,"TYPE_NAME":"integer","COLUMN_SIZE":10,"BUFFER_LENGTH":0,"DECIMAL_DIGITS":0,"NUM_PREC_RADIX":10,"NULLABLE":1,"REMARKS":null,"COLUMN_DEF":null,"SQL_DATA_TYPE":2,"SQL_DATETIME_SUB":0,"CHAR_OCTET_LENGTH":0,"ORDINAL_POSITION":8,"IS_NULLABLE":"YES","SCOPE_CATALOG":null,"SCOPE_SCHEMA":null,"SCOPE_TABLE":null,"SOURCE_DATA_TYPE":null,"IS_AUTOINCREMENT":"NO"},"division":{"TABLE_CAT":"i_l01","TABLE_SCHEM":"isnt","TABLE_NAME":"zemp_sales_by_item_for_report","COLUMN_NAME":"division","DATA_TYPE":12,"TYPE_NAME":"varchar","COLUMN_SIZE":3,"BUFFER_LENGTH":0,"DECIMAL_DIGITS":0,"NUM_PREC_RADIX":0,"NULLABLE":1,"REMARKS":null,"COLUMN_DEF":null,"SQL_DATA_TYPE":13,"SQL_DATETIME_SUB":0,"CHAR_OCTET_LENGTH":3,"ORDINAL_POSITION":9,"IS_NULLABLE":"YES","SCOPE_CATALOG":null,"SCOPE_SCHEMA":null,"SCOPE_TABLE":null,"SOURCE_DATA_TYPE":null,"IS_AUTOINCREMENT":"NO"},"department":{"TABLE_CAT":"i_l01","TABLE_SCHEM":"isnt","TABLE_NAME":"zemp_sales_by_item_for_report","COLUMN_NAME":"department","DATA_TYPE":12,"TYPE_NAME":"varchar","COLUMN_SIZE":3,"BUFFER_LENGTH":0,"DECIMAL_DIGITS":0,"NUM_PREC_RADIX":0,"NULLABLE":1,"REMARKS":null,"COLUMN_DEF":null,"SQL_DATA_TYPE":13,"SQL_DATETIME_SUB":0,"CHAR_OCTET_LENGTH":3,"ORDINAL_POSITION":10,"IS_NULLABLE":"YES","SCOPE_CATALOG":null,"SCOPE_SCHEMA":null,"SCOPE_TABLE":null,"SOURCE_DATA_TYPE":null,"IS_AUTOINCREMENT":"NO"},"category":{"TABLE_CAT":"i_l01","TABLE_SCHEM":"isnt","TABLE_NAME":"zemp_sales_by_item_for_report","COLUMN_NAME":"category","DATA_TYPE":12,"TYPE_NAME":"varchar","COLUMN_SIZE":10,"BUFFER_LENGTH":0,"DECIMAL_DIGITS":0,"NUM_PREC_RADIX":0,"NULLABLE":1,"REMARKS":null,"COLUMN_DEF":null,"SQL_DATA_TYPE":13,"SQL_DATETIME_SUB":0,"CHAR_OCTET_LENGTH":10,"ORDINAL_POSITION":11,"IS_NULLABLE":"YES","SCOPE_CATALOG":null,"SCOPE_SCHEMA":null,"SCOPE_TABLE":null,"SOURCE_DATA_TYPE":null,"IS_AUTOINCREMENT":"NO"},"item_code":{"TABLE_CAT":"i_l01","TABLE_SCHEM":"isnt","TABLE_NAME":"zemp_sales_by_item_for_report","COLUMN_NAME":"item_code","DATA_TYPE":12,"TYPE_NAME":"varchar","COLUMN_SIZE":16,"BUFFER_LENGTH":0,"DECIMAL_DIGITS":0,"NUM_PREC_RADIX":0,"NULLABLE":1,"REMARKS":null,"COLUMN_DEF":null,"SQL_DATA_TYPE":13,"SQL_DATETIME_SUB":0,"CHAR_OCTET_LENGTH":16,"ORDINAL_POSITION":12,"IS_NULLABLE":"YES","SCOPE_CATALOG":null,"SCOPE_SCHEMA":null,"SCOPE_TABLE":null,"SOURCE_DATA_TYPE":null,"IS_AUTOINCREMENT":"NO"},"line_cost":{"TABLE_CAT":"i_l01","TABLE_SCHEM":"isnt","TABLE_NAME":"zemp_sales_by_item_for_report","COLUMN_NAME":"line_cost","DATA_TYPE":3,"TYPE_NAME":"decimal","COLUMN_SIZE":18,"BUFFER_LENGTH":0,"DECIMAL_DIGITS":2,"NUM_PREC_RADIX":10,"NULLABLE":1,"REMARKS":null,"COLUMN_DEF":null,"SQL_DATA_TYPE":5,"SQL_DATETIME_SUB":0,"CHAR_OCTET_LENGTH":0,"ORDINAL_POSITION":13,"IS_NULLABLE":"YES","SCOPE_CATALOG":null,"SCOPE_SCHEMA":null,"SCOPE_TABLE":null,"SOURCE_DATA_TYPE":null,"IS_AUTOINCREMENT":"NO"},"shipped_ex_tax":{"TABLE_CAT":"i_l01","TABLE_SCHEM":"isnt","TABLE_NAME":"zemp_sales_by_item_for_report","COLUMN_NAME":"shipped_ex_tax","DATA_TYPE":3,"TYPE_NAME":"decimal","COLUMN_SIZE":32,"BUFFER_LENGTH":0,"DECIMAL_DIGITS":2,"NUM_PREC_RADIX":10,"NULLABLE":1,"REMARKS":null,"COLUMN_DEF":null,"SQL_DATA_TYPE":5,"SQL_DATETIME_SUB":0,"CHAR_OCTET_LENGTH":0,"ORDINAL_POSITION":14,"IS_NULLABLE":"YES","SCOPE_CATALOG":null,"SCOPE_SCHEMA":null,"SCOPE_TABLE":null,"SOURCE_DATA_TYPE":null,"IS_AUTOINCREMENT":"NO"},"shipped_amount":{"TABLE_CAT":"i_l01","TABLE_SCHEM":"isnt","TABLE_NAME":"zemp_sales_by_item_for_report","COLUMN_NAME":"shipped_amount","DATA_TYPE":3,"TYPE_NAME":"decimal","COLUMN_SIZE":18,"BUFFER_LENGTH":0,"DECIMAL_DIGITS":2,"NUM_PREC_RADIX":10,"NULLABLE":1,"REMARKS":null,"COLUMN_DEF":null,"SQL_DATA_TYPE":5,"SQL_DATETIME_SUB":0,"CHAR_OCTET_LENGTH":0,"ORDINAL_POSITION":15,"IS_NULLABLE":"YES","SCOPE_CATALOG":null,"SCOPE_SCHEMA":null,"SCOPE_TABLE":null,"SOURCE_DATA_TYPE":null,"IS_AUTOINCREMENT":"NO"},"margin":{"TABLE_CAT":"i_l01","TABLE_SCHEM":"isnt","TABLE_NAME":"zemp_sales_by_item_for_report","COLUMN_NAME":"margin","DATA_TYPE":3,"TYPE_NAME":"decimal","COLUMN_SIZE":18,"BUFFER_LENGTH":0,"DECIMAL_DIGITS":2,"NUM_PREC_RADIX":10,"NULLABLE":1,"REMARKS":null,"COLUMN_DEF":null,"SQL_DATA_TYPE":5,"SQL_DATETIME_SUB":0,"CHAR_OCTET_LENGTH":0,"ORDINAL_POSITION":16,"IS_NULLABLE":"YES","SCOPE_CATALOG":null,"SCOPE_SCHEMA":null,"SCOPE_TABLE":null,"SOURCE_DATA_TYPE":null,"IS_AUTOINCREMENT":"NO"},"margin_percentage":{"TABLE_CAT":"i_l01","TABLE_SCHEM":"isnt","TABLE_NAME":"zemp_sales_by_item_for_report","COLUMN_NAME":"margin_percentage","DATA_TYPE":3,"TYPE_NAME":"decimal","COLUMN_SIZE":32,"BUFFER_LENGTH":0,"DECIMAL_DIGITS":2,"NUM_PREC_RADIX":10,"NULLABLE":1,"REMARKS":null,"COLUMN_DEF":null,"SQL_DATA_TYPE":5,"SQL_DATETIME_SUB":0,"CHAR_OCTET_LENGTH":0,"ORDINAL_POSITION":17,"IS_NULLABLE":"YES","SCOPE_CATALOG":null,"SCOPE_SCHEMA":null,"SCOPE_TABLE":null,"SOURCE_DATA_TYPE":null,"IS_AUTOINCREMENT":"NO"},"isbulk":{"TABLE_CAT":"i_l01","TABLE_SCHEM":"isnt","TABLE_NAME":"zemp_sales_by_item_for_report","COLUMN_NAME":"isbulk","DATA_TYPE":4,"TYPE_NAME":"integer","COLUMN_SIZE":10,"BUFFER_LENGTH":0,"DECIMAL_DIGITS":0,"NUM_PREC_RADIX":10,"NULLABLE":1,"REMARKS":null,"COLUMN_DEF":null,"SQL_DATA_TYPE":2,"SQL_DATETIME_SUB":0,"CHAR_OCTET_LENGTH":0,"ORDINAL_POSITION":18,"IS_NULLABLE":"YES","SCOPE_CATALOG":null,"SCOPE_SCHEMA":null,"SCOPE_TABLE":null,"SOURCE_DATA_TYPE":null,"IS_AUTOINCREMENT":"NO"},"iscountedforinvoice":{"TABLE_CAT":"i_l01","TABLE_SCHEM":"isnt","TABLE_NAME":"zemp_sales_by_item_for_report","COLUMN_NAME":"iscountedforinvoice","DATA_TYPE":4,"TYPE_NAME":"integer","COLUMN_SIZE":10,"BUFFER_LENGTH":0,"DECIMAL_DIGITS":0,"NUM_PREC_RADIX":10,"NULLABLE":1,"REMARKS":null,"COLUMN_DEF":null,"SQL_DATA_TYPE":2,"SQL_DATETIME_SUB":0,"CHAR_OCTET_LENGTH":0,"ORDINAL_POSITION":19,"IS_NULLABLE":"YES","SCOPE_CATALOG":null,"SCOPE_SCHEMA":null,"SCOPE_TABLE":null,"SOURCE_DATA_TYPE":null,"IS_AUTOINCREMENT":"NO"},"class":{"TABLE_CAT":"i_l01","TABLE_SCHEM":"isnt","TABLE_NAME":"zemp_sales_by_item_for_report","COLUMN_NAME":"class","DATA_TYPE":12,"TYPE_NAME":"varchar","COLUMN_SIZE":4,"BUFFER_LENGTH":0,"DECIMAL_DIGITS":0,"NUM_PREC_RADIX":0,"NULLABLE":1,"REMARKS":null,"COLUMN_DEF":null,"SQL_DATA_TYPE":13,"SQL_DATETIME_SUB":0,"CHAR_OCTET_LENGTH":4,"ORDINAL_POSITION":20,"IS_NULLABLE":"YES","SCOPE_CATALOG":null,"SCOPE_SCHEMA":null,"SCOPE_TABLE":null,"SOURCE_DATA_TYPE":null,"IS_AUTOINCREMENT":"NO"},"item_group":{"TABLE_CAT":"i_l01","TABLE_SCHEM":"isnt","TABLE_NAME":"zemp_sales_by_item_for_report","COLUMN_NAME":"item_group","DATA_TYPE":12,"TYPE_NAME":"varchar","COLUMN_SIZE":4,"BUFFER_LENGTH":0,"DECIMAL_DIGITS":0,"NUM_PREC_RADIX":0,"NULLABLE":1,"REMARKS":null,"COLUMN_DEF":null,"SQL_DATA_TYPE":13,"SQL_DATETIME_SUB":0,"CHAR_OCTET_LENGTH":4,"ORDINAL_POSITION":21,"IS_NULLABLE":"YES","SCOPE_CATALOG":null,"SCOPE_SCHEMA":null,"SCOPE_TABLE":null,"SOURCE_DATA_TYPE":null,"IS_AUTOINCREMENT":"NO"},"item_description":{"TABLE_CAT":"i_l01","TABLE_SCHEM":"isnt","TABLE_NAME":"zemp_sales_by_item_for_report","COLUMN_NAME":"item_description","DATA_TYPE":12,"TYPE_NAME":"varchar","COLUMN_SIZE":30,"BUFFER_LENGTH":0,"DECIMAL_DIGITS":0,"NUM_PREC_RADIX":0,"NULLABLE":1,"REMARKS":null,"COLUMN_DEF":null,"SQL_DATA_TYPE":13,"SQL_DATETIME_SUB":0,"CHAR_OCTET_LENGTH":30,"ORDINAL_POSITION":22,"IS_NULLABLE":"YES","SCOPE_CATALOG":null,"SCOPE_SCHEMA":null,"SCOPE_TABLE":null,"SOURCE_DATA_TYPE":null,"IS_AUTOINCREMENT":"NO"},"so_order_status":{"TABLE_CAT":"i_l01","TABLE_SCHEM":"isnt","TABLE_NAME":"zemp_sales_by_item_for_report","COLUMN_NAME":"so_order_status","DATA_TYPE":12,"TYPE_NAME":"varchar","COLUMN_SIZE":2,"BUFFER_LENGTH":0,"DECIMAL_DIGITS":0,"NUM_PREC_RADIX":0,"NULLABLE":1,"REMARKS":null,"COLUMN_DEF":null,"SQL_DATA_TYPE":13,"SQL_DATETIME_SUB":0,"CHAR_OCTET_LENGTH":2,"ORDINAL_POSITION":23,"IS_NULLABLE":"YES","SCOPE_CATALOG":null,"SCOPE_SCHEMA":null,"SCOPE_TABLE":null,"SOURCE_DATA_TYPE":null,"IS_AUTOINCREMENT":"NO"},"sol_price_rule":{"TABLE_CAT":"i_l01","TABLE_SCHEM":"isnt","TABLE_NAME":"zemp_sales_by_item_for_report","COLUMN_NAME":"sol_price_rule","DATA_TYPE":12,"TYPE_NAME":"varchar","COLUMN_SIZE":2,"BUFFER_LENGTH":0,"DECIMAL_DIGITS":0,"NUM_PREC_RADIX":0,"NULLABLE":1,"REMARKS":null,"COLUMN_DEF":null,"SQL_DATA_TYPE":13,"SQL_DATETIME_SUB":0,"CHAR_OCTET_LENGTH":2,"ORDINAL_POSITION":24,"IS_NULLABLE":"YES","SCOPE_CATALOG":null,"SCOPE_SCHEMA":null,"SCOPE_TABLE":null,"SOURCE_DATA_TYPE":null,"IS_AUTOINCREMENT":"NO"},"sol_line_seq":{"TABLE_CAT":"i_l01","TABLE_SCHEM":"isnt","TABLE_NAME":"zemp_sales_by_item_for_report","COLUMN_NAME":"sol_line_seq","DATA_TYPE":3,"TYPE_NAME":"decimal","COLUMN_SIZE":7,"BUFFER_LENGTH":0,"DECIMAL_DIGITS":2,"NUM_PREC_RADIX":10,"NULLABLE":1,"REMARKS":null,"COLUMN_DEF":null,"SQL_DATA_TYPE":5,"SQL_DATETIME_SUB":0,"CHAR_OCTET_LENGTH":0,"ORDINAL_POSITION":25,"IS_NULLABLE":"YES","SCOPE_CATALOG":null,"SCOPE_SCHEMA":null,"SCOPE_TABLE":null,"SOURCE_DATA_TYPE":null,"IS_AUTOINCREMENT":"NO"},"sol_shipped_qty":{"TABLE_CAT":"i_l01","TABLE_SCHEM":"isnt","TABLE_NAME":"zemp_sales_by_item_for_report","COLUMN_NAME":"sol_shipped_qty","DATA_TYPE":3,"TYPE_NAME":"decimal","COLUMN_SIZE":28,"BUFFER_LENGTH":0,"DECIMAL_DIGITS":11,"NUM_PREC_RADIX":10,"NULLABLE":1,"REMARKS":null,"COLUMN_DEF":null,"SQL_DATA_TYPE":5,"SQL_DATETIME_SUB":0,"CHAR_OCTET_LENGTH":0,"ORDINAL_POSITION":26,"IS_NULLABLE":"YES","SCOPE_CATALOG":null,"SCOPE_SCHEMA":null,"SCOPE_TABLE":null,"SOURCE_DATA_TYPE":null,"IS_AUTOINCREMENT":"NO"},"so_whse_code_ori":{"TABLE_CAT":"i_l01","TABLE_SCHEM":"isnt","TABLE_NAME":"zemp_sales_by_item_for_report","COLUMN_NAME":"so_whse_code_ori","DATA_TYPE":12,"TYPE_NAME":"varchar","COLUMN_SIZE":4,"BUFFER_LENGTH":0,"DECIMAL_DIGITS":0,"NUM_PREC_RADIX":0,"NULLABLE":1,"REMARKS":null,"COLUMN_DEF":null,"SQL_DATA_TYPE":13,"SQL_DATETIME_SUB":0,"CHAR_OCTET_LENGTH":4,"ORDINAL_POSITION":27,"IS_NULLABLE":"YES","SCOPE_CATALOG":null,"SCOPE_SCHEMA":null,"SCOPE_TABLE":null,"SOURCE_DATA_TYPE":null,"IS_AUTOINCREMENT":"NO"},"cust_group":{"TABLE_CAT":"i_l01","TABLE_SCHEM":"isnt","TABLE_NAME":"zemp_sales_by_item_for_report","COLUMN_NAME":"cust_group","DATA_TYPE":12,"TYPE_NAME":"varchar","COLUMN_SIZE":4,"BUFFER_LENGTH":0,"DECIMAL_DIGITS":0,"NUM_PREC_RADIX":0,"NULLABLE":1,"REMARKS":null,"COLUMN_DEF":null,"SQL_DATA_TYPE":13,"SQL_DATETIME_SUB":0,"CHAR_OCTET_LENGTH":4,"ORDINAL_POSITION":28,"IS_NULLABLE":"YES","SCOPE_CATALOG":null,"SCOPE_SCHEMA":null,"SCOPE_TABLE":null,"SOURCE_DATA_TYPE":null,"IS_AUTOINCREMENT":"NO"},"cust_category":{"TABLE_CAT":"i_l01","TABLE_SCHEM":"isnt","TABLE_NAME":"zemp_sales_by_item_for_report","COLUMN_NAME":"cust_category","DATA_TYPE":12,"TYPE_NAME":"varchar","COLUMN_SIZE":4,"BUFFER_LENGTH":0,"DECIMAL_DIGITS":0,"NUM_PREC_RADIX":0,"NULLABLE":1,"REMARKS":null,"COLUMN_DEF":null,"SQL_DATA_TYPE":13,"SQL_DATETIME_SUB":0,"CHAR_OCTET_LENGTH":4,"ORDINAL_POSITION":29,"IS_NULLABLE":"YES","SCOPE_CATALOG":null,"SCOPE_SCHEMA":null,"SCOPE_TABLE":null,"SOURCE_DATA_TYPE":null,"IS_AUTOINCREMENT":"NO"},"cust_segment":{"TABLE_CAT":"i_l01","TABLE_SCHEM":"isnt","TABLE_NAME":"zemp_sales_by_item_for_report","COLUMN_NAME":"cust_segment","DATA_TYPE":12,"TYPE_NAME":"varchar","COLUMN_SIZE":4,"BUFFER_LENGTH":0,"DECIMAL_DIGITS":0,"NUM_PREC_RADIX":0,"NULLABLE":1,"REMARKS":null,"COLUMN_DEF":null,"SQL_DATA_TYPE":13,"SQL_DATETIME_SUB":0,"CHAR_OCTET_LENGTH":4,"ORDINAL_POSITION":30,"IS_NULLABLE":"YES","SCOPE_CATALOG":null,"SCOPE_SCHEMA":null,"SCOPE_TABLE":null,"SOURCE_DATA_TYPE":null,"IS_AUTOINCREMENT":"NO"},"cust_subsegment":{"TABLE_CAT":"i_l01","TABLE_SCHEM":"isnt","TABLE_NAME":"zemp_sales_by_item_for_report","COLUMN_NAME":"cust_subsegment","DATA_TYPE":12,"TYPE_NAME":"varchar","COLUMN_SIZE":4,"BUFFER_LENGTH":0,"DECIMAL_DIGITS":0,"NUM_PREC_RADIX":0,"NULLABLE":1,"REMARKS":null,"COLUMN_DEF":null,"SQL_DATA_TYPE":13,"SQL_DATETIME_SUB":0,"CHAR_OCTET_LENGTH":4,"ORDINAL_POSITION":31,"IS_NULLABLE":"YES","SCOPE_CATALOG":null,"SCOPE_SCHEMA":null,"SCOPE_TABLE":null,"SOURCE_DATA_TYPE":null,"IS_AUTOINCREMENT":"NO"},"del_method":{"TABLE_CAT":"i_l01","TABLE_SCHEM":"isnt","TABLE_NAME":"zemp_sales_by_item_for_report","COLUMN_NAME":"del_method","DATA_TYPE":12,"TYPE_NAME":"varchar","COLUMN_SIZE":10,"BUFFER_LENGTH":0,"DECIMAL_DIGITS":0,"NUM_PREC_RADIX":0,"NULLABLE":1,"REMARKS":null,"COLUMN_DEF":null,"SQL_DATA_TYPE":13,"SQL_DATETIME_SUB":0,"CHAR_OCTET_LENGTH":10,"ORDINAL_POSITION":32,"IS_NULLABLE":"YES","SCOPE_CATALOG":null,"SCOPE_SCHEMA":null,"SCOPE_TABLE":null,"SOURCE_DATA_TYPE":null,"IS_AUTOINCREMENT":"NO"},"member_type":{"TABLE_CAT":"i_l01","TABLE_SCHEM":"isnt","TABLE_NAME":"zemp_sales_by_item_for_report","COLUMN_NAME":"member_type","DATA_TYPE":1,"TYPE_NAME":"char","COLUMN_SIZE":2,"BUFFER_LENGTH":0,"DECIMAL_DIGITS":0,"NUM_PREC_RADIX":0,"NULLABLE":1,"REMARKS":null,"COLUMN_DEF":null,"SQL_DATA_TYPE":0,"SQL_DATETIME_SUB":0,"CHAR_OCTET_LENGTH":2,"ORDINAL_POSITION":33,"IS_NULLABLE":"YES","SCOPE_CATALOG":null,"SCOPE_SCHEMA":null,"SCOPE_TABLE":null,"SOURCE_DATA_TYPE":null,"IS_AUTOINCREMENT":"NO"}}}
read_fwf('fn'  ,header=3,
names=['so_invoice_date', 'ag_qty', 'available_stock', 'item_code', 'so_whse_code', 'division', 'department', 'category', 'class', 'item_group', 'cust_group', 'cust_category', 'cust_segment', 'cust_subsegment', 'sol_price_rule', 'del_method'],
colspecs=[(0, 10), (10, 42), (42, 62), (62, 78), (78, 82), (82, 85), (85, 88), (88, 98), (98, 102), (102, 106), (106, 110), (110, 114), (114, 118), (118, 122), (122, 124), (124, 134)])
asfimport commented 5 years ago

Jim Northrup: for the record we have pre-tensorflow column counts of about 14000 one-hot attributes. we are seeing numpy RAM requirements of 160 gigs

asfimport commented 5 years ago

Micah Kornfield / @emkornfield: Thanks for the update.  The code on master might still have memory issues, because it tries to load all records in one contiguous batch but with the memory leak fixed it might allow you to move forward.  I filed https://issues.apache.org/jira/browse/ARROW-6219 to track improving the API so result sets can be batched up.

 

"for the record we have pre-tensorflow column counts of about 14000 one-hot attributes. we are seeing numpy RAM requirements of 160 gigs"

The current java implementation is purely off-heap and the current allocator support Integer.MAX_INT bytes. Technically, a separate allocator per column could be used but I don't think that is a great solution here.

There is a discussion on the mailing list about supporting 64-bit address spaces (https://lists.apache.org/thread.html/dcd1ae4d943b40568e6b178b91cb7ed012168711f9eb2bbf3c3cbd2d@%3Cdev.arrow.apache.org%3E).

14000 columns might stretch the arrow format a little bit.  There was previous discussions on the mailing list about tensor modeling (https://lists.apache.org/thread.html/8e48930c39bf9b70f28f9a83d1b7e9e71fa45d85d711e25dedc598f9@%3Cdev.arrow.apache.org%3E) that might be worth looking at.

 

asfimport commented 5 years ago

Micah Kornfield / @emkornfield: Sorry should clarify above. With an 800mb of data I think master should be fixed.   I will try to verify tonight

asfimport commented 5 years ago

Micah Kornfield / @emkornfield: OK, so even with the old API due to how the allocator works it seems unlikely to work.  The good news is with the new iterator API you can processing the result set automatically. 

asfimport commented 5 years ago

Bryan Cutler / @BryanCutler: Issue resolved by pull request 5134 https://github.com/apache/arrow/pull/5134