deephaven / deephaven-core

Deephaven Community Core
Other
254 stars 80 forks source link

AppMode with QueryString #1372

Closed hythloda closed 2 years ago

hythloda commented 3 years ago

Description

Something with QS is not working with lists.

This is silly looking not in the dynamic table but shows the problem without complexity.

This is my .py script inside app.d

from deephaven import TableTools
from typing import Callable
import jpy
from deephaven.TableTools import newTable, stringCol

ApplicationState = jpy.get_type('io.deephaven.appmode.ApplicationState')

#works
ids=['bitcoin','cardano']

def table_func(app: ApplicationState):
    for names in ids:
        #This works
        table1 = newTable(
        stringCol("Strings", ids))
        #adding to it in QS does not work
        result = table1.view("DateTime =  names")
    app.setField("HistoricalCryptoTable", result)

def initialize(func: Callable[[ApplicationState], None]):
  app = jpy.get_type('io.deephaven.appmode.ApplicationContext').get()
  func(app)

initialize(table_func)

Expected results

See a table with 2 columns and 2 rows.

Actual results

Error in terminal after running app mode: Exception message : Cannot find variable or class names

This works inside the console:

ids=['bitcoin','cardano']

for names in ids:
    #This works
    table1 = newTable(
    stringCol("Strings", ids))
    result = table1.view("DateTime =  names")

For full example working and broke see attached zips. run ./start.sh from directory to launch

hythloda commented 3 years ago

BROKEN_cryptocurrency.zip WORKING_cryptocurrency.zip

hythloda commented 3 years ago

Another intersting thing that is related is that this code does not work in app mode but does in the console in python, app mode gives error Value: invalid character in identifier (<string>, line 6):

from deephaven.TableTools import timeTable
from deephaven import SortPair
from deephaven import ComboAggregateFactory as caf
from deephaven.conversion_utils import NULL_DOUBLE
import random
Symbols = [“SPY”, “PFF”, “FB”, “DIS”, “USO”, “XLF”, “XLU”, “RBLX”, “REM”, “REZ”, “GME”, “HDSN”]
Exchanges = [“NYSE”, “INET”, “EDGX”, “BATS”, “EDGA” , “MEMX”, “BATY”, “XOTC”, “OTCU”]
priceMap = {“PFF”:37.50, “SPY”:395.00, “USO”:40.00, “XLF”:31.50, “XLU”:62.50, “REM”:33.33, “REZ”:71.33, “HDSN”:1.85, “FB”:270.00, “DIS”:190.00, “GME”:17.50, “RBLX”:70}
def pricerFunc (USym,  E):
   _aPrice = priceMap[USym]
   if(_aPrice ==  NULL_DOUBLE):
       priceMap[USym] = abs(E)/100.0
   else:
       priceMap[USym] = (_aPrice*100 + (int)(E/2000))/100.0
   return priceMap[USym]
def distributerFunc(cnt):
    n = random.randint(0, (int)(cnt*(cnt+1))/2-1) +1
    for i in range(cnt):
      n = n - (cnt-i)
      if (n <= 0):
          return int(i)

x = len(Symbols)
y = len(Exchanges)
TickingTable = timeTable('00:00:00.013').tail(100000).update('A=random.randint(0,10)', 'B=random.randint(0,2)', 'C=random.randint(0,50)', 'D= ((int) (random.randint(0,100))/100.0 - 0.5) * 20000.0', 'USym=Symbols[((int)distributerFunc(x))-1]', 'Size=Math.max(((int) random.randint(0, 11))*100, ((int) random.randint(0, 149))+1)', 'Price=(double)pricerFunc(USym, D)', 'TradeVal=Size*Price', 'Exchange = Exchanges[(int)C%y]', 'I=ii').dropColumns('C', 'D')
Cumulative = TickingTable.update('TradeVal = Size * Price').by(caf.AggCombo(caf.AggSum('Volume = Size', 'TradeVal'), caf.AggWAvg('Size', 'VWAP=Price')), 'USym', 'Exchange').sort(SortPair.descending('TradeVal'))

But replacing the double quotes with singles allows it to function, in the console both are accepted. This works in app mode:

from deephaven.TableTools import timeTable
from deephaven import SortPair
from deephaven import ComboAggregateFactory as caf
from deephaven.conversion_utils import NULL_DOUBLE
import random
Symbols = ['SPY', 'PFF', 'FB', 'DIS', 'USO', 'XLF', 'XLU', 'RBLX', 'REM', 'REZ', 'GME', 'HDSN']
Exchanges = ['NYSE', 'INET', 'EDGX', 'BATS', 'EDGA' , 'MEMX', 'BATY', 'XOTC', 'OTCU']
priceMap = {'PFF':37.50, 'SPY':395.00, 'USO':40.00, 'XLF':31.50, 'XLU':62.50, 'REM':33.33, 'REZ':71.33, 'HDSN':1.85, 'FB':270.00, 'DIS':190.00, 'GME':17.50, 'RBLX':70}
def pricerFunc (USym,  E):
   _aPrice = priceMap[USym]
   if(_aPrice ==  NULL_DOUBLE):
       priceMap[USym] = abs(E)/100.0
   else:
       priceMap[USym] = (_aPrice*100 + (int)(E/2000))/100.0
   return priceMap[USym]
def distributerFunc(cnt):
    n = random.randint(0, (int)(cnt*(cnt+1))/2-1) +1
    for i in range(cnt):
      n = n - (cnt-i)
      if (n <= 0):
          return int(i)

x = len(Symbols)
y = len(Exchanges)
TickingTable = timeTable('00:00:00.013').tail(100000).update('A=random.randint(0,10)', 'B=random.randint(0,2)', 'C=random.randint(0,50)', 'D= ((int) (random.randint(0,100))/100.0 - 0.5) * 20000.0', 'USym=Symbols[((int)distributerFunc(x))-1]', 'Size=Math.max(((int) random.randint(0, 11))*100, ((int) random.randint(0, 149))+1)', 'Price=(double)pricerFunc(USym, D)', 'TradeVal=Size*Price', 'Exchange = Exchanges[(int)C%y]', 'I=ii').dropColumns('C', 'D')
Cumulative = TickingTable.update('TradeVal = Size * Price').by(caf.AggCombo(caf.AggSum('Volume = Size', 'TradeVal'), caf.AggWAvg('Size', 'VWAP=Price')), 'USym', 'Exchange').sort(SortPair.descending('TradeVal'))
nbauernfeind commented 3 years ago

I believe this is caused by the first example using smart quotes instead of regular quotes. Can you confirm?

nbauernfeind commented 2 years ago

The issue is that inside of the method names is only a local variable. Local variables are not accessible to the query scope.

The work around is to declare names global prior to using it in the for loop.

def table_func(app: ApplicationState):
    global names
    for names in ids:
        table1 = new_table([
          string_col("Strings", ids)
        ])
        result = table1.view("DateTime =  names")
    app.setField("HistoricalCryptoTable", result)