[BUG] - 2_live_data - chart.update(series) not working

[qarampage]

Expected Behavior

Hi @louisnw01 /Larry,

I have just checked out your code. and executed it with the existing dataset you have and did not make any change to the code. I got the error at chart.update(series) from df2 , where df2 is next_ohlcv.csv It should load the series and display on the chart

Current Behaviour

There is an error

Traceback (most recent call last): File "pandas/_libs/tslib.pyx", line 304, in pandas._libs.tslib.array_with_unit_to_datetime ValueError: could not convert string to float: '2016-12-22' During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/Users/lightweight-charts-python/examples/2_live_data/live_data.py", line 19, in chart.update(series) File "/Users/myname/projects/pyvenv/lib/python3.9/site-packages/lightweight_charts/abstract.py", line 596, in update series = self._series_datetime_format(series) if not _from_tick else series File "/Users/myname/projects/pyvenv/lib/python3.9/site-packages/lightweight_charts/abstract.py", line 208, in _series_datetime_format series['time'] = self._single_datetime_format(series['time']) File "/Users/myname/projects/pyvenv/lib/python3.9/site-packages/lightweight_charts/abstract.py", line 213, in _single_datetime_format arg = pd.to_datetime(arg, unit='ms') File "/Users/myname/projects/pyvenv/lib/python3.9/site-packages/pandas/core/tools/datetimes.py", line 1084, in to_datetime result = convert_listlike(np.array([arg]), format)[0] File "/Users/myname/projects/pyvenv/lib/python3.9/site-packages/pandas/core/tools/datetimes.py", line 427, in _convert_listlike_datetimes return _to_datetime_with_unit(arg, unit, name, utc, errors) File "/Users/myname/projects/pyvenv/lib/python3.9/site-packages/pandas/core/tools/datetimes.py", line 539, in _to_datetime_with_unit arr, tz_parsed = tslib.array_with_unit_to_datetime(arg, unit, errors=errors) File "pandas/_libs/tslib.pyx", line 332, in pandas._libs.tslib.array_with_unit_to_datetime File "pandas/_libs/tslib.pyx", line 306, in pandas._libs.tslib.array_with_unit_to_datetime ValueError: non convertible value 2016-12-22 with the unit 'ms', at position 0

Reproducible Example

I have executed your code without making any change to it

the exact location where it failed is the lightweight_chart -> abstract.py in the function below at line 
 arg = pd.to_datetime(arg, unit='ms')

    def _single_datetime_format(self, arg):
        if isinstance(arg, (str, int, float)) or not pd.api.types.is_datetime64_any_dtype(arg):
            arg = pd.to_datetime(arg, unit='ms')
        arg = self._interval * (arg.timestamp() // self._interval)+self.offset
        return arg

Environment

- OS: Mac 14.2.1 (23C71)

[qarampage]

And I have this data in my dataset

date,open,high,low,close,volume 20/12/23 9:15,183.1,183.1,183.1,186.05,0 20/12/23 9:16,186.05,223,186.05,223,150 20/12/23 9:17,223,225,223,225,0 20/12/23 9:18,225,228.9,222.7,228.9,350 20/12/23 9:19,228.9,228.9,225.1,225.1,150 20/12/23 9:20,225.1,225.1,225.1,225.1,0 20/12/23 9:21,225.1,228,225.1,228,150 20/12/23 9:22,228,232.95,228,232.95,500 ...... .......

When I executed this code : it showed me a blank chart that's your setting_data.py chart = Chart()

# Columns: time | open | high | low | close | volume
df = pd.read_csv('NIFTY2411121800CE_sample.csv')

# Added code to convert the date which I have into the one that you use in ohlc.csv
df['date'] = pd.to_datetime(df['date'], format='%y/%m/%d %H:%M')
df['date'] = df['date'].dt.strftime('%Y-%m-%d %H:%M')
# ----  code to convert the date which I have into the one that you use in ohlcv.csv ---

chart.set(df)
chart.show(block=True)

[cilindro5432]

try this version works like a charm lightweightcharts copy.zip

[louisnw01]

All fixed in the latest version.

Louis

[qarampage]

@louisnw01 - it is not working with your data also . it doesn't work wit the example dataset I provided in my prev comment. I have used the code in ZIP file also. It doesn't with your example data set. here is the error while triggering chart.update(series). The first part gets loaded but fail in the next part

ValueError: could not convert string to float: '2016-12-22'

Sample dataset and format time,open,high,low,close,volume 2024-01-01 14:00:00,297.8,316.4,297.55,315.3,1350

[louisnw01]

@louisnw01 - it is not working with your data also . it doesn't work wit the example dataset I provided in my prev comment.

I have used the code in ZIP file also. It doesn't with your example data set. here is the error while triggering

chart.update(series). The first part gets loaded but fail in the next part

ValueError: could not convert string to float: '2016-12-22'

Sample dataset and format

time,open,high,low,close,volume

2024-01-01 14:00:00,297.8,316.4,297.55,315.3,1350

Can you confirm you're using the latest version? (pip install --upgrade lightweight-charts)

[qarampage]

--upgrade lightweight-charts

Hi @louisnw01 : Thanks so much. after upgrading lightweight-charts, it worked with your dataset. I came across your code few days back only and installed lightweight-charts for the 1st time. anyways. it is working fine with your dataset. So, Today I just upgraded lightweight-charts and took an update from your main branch which is at [f31db04]

But I want to now make best use of this library to use the data set I have in the following format below . Could you please help me out? I am not able to use the 2_live_data file with my data set . All the data I have has below datetime format 2024-01-01 14:15:00

Example Data set I am using time,open,high,low,close,volume 2024-01-01 14:15:00,297.8,316.4,297.55,315.3,1350

I am using a 3 min data set of past 30 days . divided into 2 files. 15 days each. When I run your code 2_live_data code with following changes , I get Error when it triggers the chart.update(series). However important to note is that it is able to load the initial data using chart.set(df1)

1st Attempt - fails at chart.update(series)

if __name__ == '__main__':
    chart = Chart()
    df1 = pd.read_csv('NIFTY2411821800CE_sample_p1.csv')
    df2 = pd.read_csv('NIFTY2411821800CE_sample_p2.csv')
    chart.set(df1)
    chart.show()
    last_close = df1.iloc[-1]
    for i, series in df2.iterrows():
        chart.update(series)
        if series['close'] > 20 and last_close < 20:
            chart.marker(text='The price crossed $20!')
        last_close = series['close']
        sleep(0.1)

Traceback (most recent call last): File "/Users/projects/others/mlpython/trading/lightweight-charts-python/examples/2_live_data/live_data_fixed.py", line 23, in if series['close'] > 20 and last_close < 20: File "/Users/projects/pyvenv/lib/python3.9/site-packages/pandas/core/ops/common.py", line 81, in new_method return method(self, other) File "/Users/projects/pyvenv/lib/python3.9/site-packages/pandas/core/arraylike.py", line 48, in lt return self._cmp_method(other, operator.lt) File "/Users/projects/pyvenv/lib/python3.9/site-packages/pandas/core/series.py", line 6092, in _cmp_method res_values = ops.comparison_op(lvalues, rvalues, op) File "/Users/projects/pyvenv/lib/python3.9/site-packages/pandas/core/ops/array_ops.py", line 293, in comparison_op res_values = comp_method_OBJECT_ARRAY(op, lvalues, rvalues) File "/Users/projects/pyvenv/lib/python3.9/site-packages/pandas/core/ops/array_ops.py", line 82, in comp_method_OBJECT_ARRAY result = libops.scalar_compare(x.ravel(), y, op) File "pandas/_libs/ops.pyx", line 107, in pandas._libs.ops.scalar_compare TypeError: '<' not supported between instances of 'str' and 'int'

2nd Attempt - with same error as above

if __name__ == '__main__':
    chart = Chart()

    **df1 = pd.read_csv('NIFTY2411821800CE_sample_p1.csv')
    df2 = pd.read_csv('NIFTY2411821800CE_sample_p2.csv')
    df1['time'] = pd.to_datetime(df1['time'], format='%Y-%m-%d %H:%M:%S')
    df1['time'] = df1['time'].dt.strftime('%Y-%m-%d %H:%M')
    df2['time'] = pd.to_datetime(df2['time'], format='%Y-%m-%d %H:%M:%S')
    df2['time'] = df2['time'].dt.strftime('%Y-%m-%d %H:%M')**
    df2.set_index('time')
    chart.set(df1)
    chart.show()
    last_close = df1.iloc[-1]
    for i, series in df2.iterrows():
        chart.update(series)
        if series['close'] > 20 and last_close < 20:
            chart.marker(text='The price crossed $20!')
        last_close = series['close']
        sleep(0.1)

3rd Attempt - fails with same error at chart.update(series)

    df1 = pd.read_csv('NIFTY2411821800CE_sample_p1.csv')
    df2 = pd.read_csv('NIFTY2411821800CE_sample_p2.csv')

    df1['time'] = pd.to_datetime(df1['time'], format='%Y-%m-%d %H:%M:%S')
    df1['time'] = df1['time'].dt.strftime('%Y-%m-%d %H:%M:**%S**')

    df2['time'] = pd.to_datetime(df2['time'], format='%Y-%m-%d %H:%M:%S')
    df2['time'] = df2['time'].dt.strftime('%Y-%m-%d %H:%M:**%S**')

here is the snapshot for your reference of df1 being displayed, df2 dataset sample and 1st series of df2

[qarampage]

@louisnw01 , need help in solving the above issue with live data script

[louisnw01]

line 23, in if series['close'] > 20 and last_close < 20: TypeError: '<' not supported between instances of 'str' and 'int'

One is str, one is int.

[qarampage]

line 23, in if series['close'] > 20 and last_close < 20: TypeError: '<' not supported between instances of 'str' and 'int' One is str, one is int.

@louisnw01 Yes I got that . but look at the snapshot below: why is it treating the data in both the Series - series and last_close differently? why is it trying to compare string and int ?

And If I run the data set you have in the 2_live_data. I get same error comparing string with int.: Please check the snapshot below of debug

@louisnw01 : finally what I did was this and it worked and properly showed the marker. Seems like you forgot to upload the that piece of code last_close = df1['close'].iloc[-1]

[qarampage]

@louisnw01: your code is not convenient to use. It not like a general purpose tool. Can't use it for the data set I have and it's coming from a broker. the format is very simple like below: time,open,high,low,close,volume 2024-01-01 14:00:00,297.8,316.4,297.55,315.3,1350

It gives json serializable error, or at time it doesn't show anything on the chart. often it have to convert the datetime column to time. makes it difficult to integrate with other code.

The code is too specific and tied to you particular data set. It's making my work difficult . I am good with basic chart matplotlib/plotly.

[louisnw01]

@louisnw01: your code is not convenient to use. It not like a general purpose tool. Can't use it for the data set I have and it's coming from a broker. the format is very simple like below:

time,open,high,low,close,volume

2024-01-01 14:00:00,297.8,316.4,297.55,315.3,1350

It gives json serializable error, or at time it doesn't show anything on the chart. often it have to convert the datetime column to time. makes it difficult to integrate with other code.

The code is too specific and tied to you particular data set. It's making my work difficult . I am good with basic chart matplotlib/plotly.

@qarampage I can't help you without a MRE.

Louis

[qarampage]

@qarampage I can't help you without a MRE. what is MRE ?

If I provide you the data I have, can you please test that with your code. because I can't get your code working with my data? Its not clear what is missing ?