search

xbrlus / xule

An open-source XBRL processor for business rules, rendering and custom data reporting. See https://xbrl.us/xule for documentation and https://xbrl.us/xule-editor for a VS Code syntax highlighter.
Apache License 2.0
24 stars 10 forks source link

Troubleshooting XULE Script Execution Error with Arelle on Amazon Linux 2 #28

Open pontsoleil opened 7 months ago

pontsoleil commented 7 months ago

I'm evaluating the XULE plugin with Arelle on Amazon Linux 2, using Python 3.8. My script, successfully executed in XMLSpy 2024, aims to analyze hierarchical XBRL instances for journal entries, using dimensions to define journal headers, lines, and sub-account details. However, while compilation appears successful, running the script with --xule-run results in an error. The script calculates debit and credit travel expenses, showing promising results when tested elsewhere:

namespace cor=http://www.iso.org/awi21926_jisc

output TravelExpense

$DebitEntries = set({covered @concept=cor:JISC04b_GL03_06 where $fact==true}.dimensions())
$CreditEntries = set({covered @concept=cor:JISC04b_GL03_06 where $fact==false}.dimensions())
$TravelExpenses = set({covered @concept=cor:JISC04b_GL03_01 where $fact=='726'}.dimensions())
$totalDebitTravelExpenses = sum(list({covered @cor:CC04w_01 where $fact.dimensions() in $DebitEntries and $fact.dimensions() in $TravelExpenses}))
$totalCreditTravelExpenses = sum(list({covered @cor:CC04w_01 where $fact.dimensions() in $CreditEntries and $fact.dimensions() in $TravelExpenses}))

"The total travel expenses are Debit:{$totalDebitTravelExpenses} JPY Credit:{$totalCreditTravelExpenses} JPY"

Successful output in XMLSpy 2023:

Successfully executed XULE on JISCinstance.xbrl in 16ms
    [TravelExpense] The total travel expenses are Debit:28,096 JPY Credit:None JPY

On Arelle, compilation feedback is positive, but execution fails:

(arelle-env) [ec2-user Arelle]$ python arelleCmdLine.py --plugins xule --xule-compile /home/ec2-user/UADC/data/xule/travel_expense.xule --xule-rule-set /home/ec2-user/UADC/data/xule/travel_expense.zip
[info] Activation of plug-in XBRL rule processor (xule) successful, version Check version using Tools->Xule->Version on the GUI or --xule-version on the command line. - xule
[info] Xule version: 3.0.23722 -
2024-03-24T14:02:47.357186: travel_expense.xule parse start
2024-03-24T14:02:47.932898: travel_expense.xule parse end. Took 0:00:00.575712
2024-03-24T14:02:47.933101: travel_expense.xule ast start
2024-03-24T14:02:47.933378: travel_expense.xule ast end. Took 0:00:00.000277
2024-03-24T14:02:47.933405: post parse start
2024-03-24T14:02:47.934987: post parse end. Took 0:00:00.001582
2024-03-24T14:02:47.936886: Parsing finished. Took 0:00:00.581420

(arelle-env) [ec2-user Arelle]$ python -m arelle.CntlrCmdLine --plugin xule --xule-run --xule-rule /home/ec2-user/UADC/data/xule/travel_expense.zip
[info] Activation of plug-in XBRL rule processor (xule) successful, version Check version using Tools->Xule->Version on the GUI or --xule-version on the command line. - xule
[info] Xule version: 3.0.23722 -
[xule:error] rule TravelExpense: 'all' -

(arelle-env) [ec2-user Arelle]$

Could you review my approach to ensure it's correct? I'm looking to refine this process before advancing to XINCE for trial balance reporting. Your insights on proceeding with this test would be greatly appreciated.

pontsoleil commented 7 months ago

On MacOS, I attempted to run a XULE rule set against an XBRL instance using Arelle and encountered an error. Here's the command I used:

$ python arelleCmdLine.py --file /Users/pontsoleil/Documents/GitHub/UADC/data/instance/JISCinstance.xbrl --plugin ~/Arelle/plugins/xule --xule-run --xule-rule /Users/pontsoleil/Documents/GitHub/UADC/XULE/JISC/travel_expense.xulec

The XULE plugin activated successfully, and the XBRL instance loaded quickly. However, during the process, I encountered the following error:

$ python arelleCmdLine.py --file /Users/pontsoleil/Documents/GitHub/UADC/data/instance/JISCinstance.xbrl --plugin ~/Arelle/plugins/xule --xule-run --xule-rule /Users/pontsoleil/Documents/GitHub/UADC/XULE/JISC/travel_expense.xulec
[info] Activation of plug-in XBRL rule processor (xule) successful, version Check version using Tools->Xule->Version on the GUI or --xule-version on the command line. - /Users/pontsoleil/Arelle/plugins/xule 
[info] Xule version: 3.0.23722 - 
[info] loaded in 0.08 secs at 2024-03-24T19:35:10 - /Users/pontsoleil/Documents/GitHub/UADC/data/instance/JISCinstance.xbrl 
[xule:error] rule TravelExpense: unhashable type: 'dict' - JISCinstance.xbrl

This suggests an issue within the TravelExpense rule in the .xulec file, potentially related to attempting to use a dictionary type in a context that requires hashable objects.

pontsoleil commented 7 months ago

I've enhanced a method for processing IDs within XULE scripts, aimed at accurately calculating travel expenses. This technique capitalizes on the substring of an ID that corresponds to a contextRef attribute, linking directly to a dimension within the XML structure. Below is an XML snippet illustrating this association:

<context id="c-02">
    <entity>
        <identifier scheme="http://www.example.com">Example Co.</identifier>
        <segment>
            <xbrldi:typedMember dimension="cor:d_JISC03">
                <cor:_v>1</cor:_v>
            </xbrldi:typedMember>
            <xbrldi:typedMember dimension="cor:d_JISC04b">
                <cor:_v>1</cor:_v>
            </xbrldi:typedMember>
        </segment>
    </entity>
    <period>
        <instant>2023-10-31</instant>
    </period>
</context>
<cor:JISC04b_GL03_01 contextRef="c-02" id="adc.r_2.JISC04b_GL03_01">726</cor:JISC04b_GL03_01>
<cor:JISC04b_GL03_02 contextRef="c-02" id="adc.r_2.JISC04b_GL03_02">Travel expense</cor:JISC04b_GL03_02>
<cor:JISC04b_GL03_06 contextRef="c-02" id="adc.r_2.JISC04b_GL03_06">true</cor:JISC04b_GL03_06>
<cor:CC04w_01 contextRef="c-02" decimals="INF" id="adc.r_2.CC04w_01" unitRef="u-01">18572</cor:CC04w_01>

The script demonstrates an efficient dimension management approach by employing ID substrings, a technique overcoming prior limitations faced with dictionary-based dimension handling. The script's operations:

Refined XULE Script

namespace cor=http://www.iso.org/awi21926_jisc

output TravelExpense

/* Sets based on conditions matched by ID substrings */
$DebitEntries = set({covered @concept=cor:JISC04b_GL03_06 where $fact==true}.id.substring(1,8))
$CreditEntries = set({covered @concept=cor:JISC04b_GL03_06 where $fact==false}.id.substring(1,8))
$TravelExpenses = set({covered @concept=cor:JISC04b_GL03_01 where $fact=='726'}.id.substring(1,8))

/* Totals calculation based on conditions */
$totalDebitTravelExpenses = sum(list({covered @cor:CC04w_01 where $fact.id.substring(1,8) in $DebitEntries and $fact.id.substring(1,8) in $TravelExpenses}))
$totalCreditTravelExpenses = sum(list({covered @cor:CC04w_01 where $fact.id.substring(1,8) in $CreditEntries and $fact.id.substring(1,8) in $TravelExpenses}))

/* Output statement for total expenses */
"The total travel expenses are Debit:{$totalDebitTravelExpenses} JPY Credit:{$totalCreditTravelExpenses} JPY"

Execution and Compilation

Utilizing Arelle and its XULE plugin on MacOS, the script underwent seamless compilation and execution.

Outcome

The script's execution verified its ability to compute total travel expenses.

Considerations for OIM-CSV

The application of this technique to OIM-CSV, which not explicitly contain ID values, remains to be clarified. The XBRL instance utilized was generated by Arelle from OIM-CSV input, highlighting the need for further investigation into the compatibility of this method with OIM-CSV formats.

campbellpryde commented 7 months ago

Could you run this command again with the option --xule-crash, so I can see the trace stack.

Campbell Pryde

President and CEO

XBRL US Inc.

@. @.>*

https://xbrl.us https://xbrl.us

[image: XBRL US]

The non-profit community https://xbrl.us/xusmembers improving U.S. reporting through a free, open standard https://xbrl.us/home/learn/free-open-standards/.

On Sun, Mar 24, 2024 at 1:37 AM SAMBUICHI, Nobuyuki < @.***> wrote:

I'm evaluating the XULE plugin with Arelle on Amazon Linux 2, using Python 3.8. My script, successfully executed in XMLSpy 2024, aims to analyze hierarchical XBRL instances for journal entries, using dimensions to define journal headers, lines, and sub-account details. However, while compilation appears successful, running the script with --xule-run results in an error. The script calculates debit and credit travel expenses, showing promising results when tested elsewhere:

namespace cor=http://www.iso.org/awi21926_jisc output TravelExpense $DebitEntries = set({covered @concept=cor:JISC04b_GL03_06 where $fact==true}.dimensions()) $CreditEntries = set({covered @concept=cor:JISC04b_GL03_06 where $fact==false}.dimensions()) $TravelExpenses = set({covered @concept=cor:JISC04b_GL03_01 where $fact=='726'}.dimensions()) $totalDebitTravelExpenses = sum(list({covered @cor:CC04w_01 where $fact.dimensions() in $DebitEntries and $fact.dimensions() in $TravelExpenses})) $totalCreditTravelExpenses = sum(list({covered @cor:CC04w_01 where $fact.dimensions() in $CreditEntries and $fact.dimensions() in $TravelExpenses}))

"The total travel expenses are Debit:{$totalDebitTravelExpenses} JPY Credit:{$totalCreditTravelExpenses} JPY"

Successful output in XMLSpy 2023:

Successfully executed XULE on JISCinstance.xbrl in 16ms [TravelExpense] The total travel expenses are Debit:28,096 JPY Credit:None JPY

On Arelle, compilation feedback is positive, but execution fails:

(arelle-env) @.*** Arelle]$ python arelleCmdLine.py --plugins xule --xule-compile /home/ec2-user/UADC/data/xule/travel_expense.xule --xule-rule-set /home/ec2-user/UADC/data/xule/travel_expense.zip [info] Activation of plug-in XBRL rule processor (xule) successful, version Check version using Tools->Xule->Version on the GUI or --xule-version on the command line. - xule [info] Xule version: 3.0.23722 - 2024-03-24T14:02:47.357186: travel_expense.xule parse start 2024-03-24T14:02:47.932898: travel_expense.xule parse end. Took 0:00:00.575712 2024-03-24T14:02:47.933101: travel_expense.xule ast start 2024-03-24T14:02:47.933378: travel_expense.xule ast end. Took 0:00:00.000277 2024-03-24T14:02:47.933405: post parse start 2024-03-24T14:02:47.934987: post parse end. Took 0:00:00.001582 2024-03-24T14:02:47.936886: Parsing finished. Took 0:00:00.581420

(arelle-env) @.*** Arelle]$ python -m arelle.CntlrCmdLine --plugin xule --xule-run --xule-rule /home/ec2-user/UADC/data/xule/travel_expense.zip [info] Activation of plug-in XBRL rule processor (xule) successful, version Check version using Tools->Xule->Version on the GUI or --xule-version on the command line. - xule [info] Xule version: 3.0.23722 - [xule:error] rule TravelExpense: 'all' -

(arelle-env) @.*** Arelle]$

Could you review my approach to ensure it's correct? I'm looking to refine this process before advancing to XINCE for trial balance reporting. Your insights on proceeding with this test would be greatly appreciated.

— Reply to this email directly, view it on GitHub https://github.com/xbrlus/xule/issues/28, or unsubscribe https://github.com/notifications/unsubscribe-auth/ADC2RF4KCQICEPR3XLQOFITYZZRCDAVCNFSM6AAAAABFFMBQWKVHI2DSMVQWIX3LMV43ASLTON2WKOZSGIYDIMJZGA2DSNY . You are receiving this because you are subscribed to this thread.Message ID: @.***>

campbellpryde commented 7 months ago

I have recreated the error and am looking at a fix for the dictionary match.