How to add new indicators to ta-lib?

[hhashim1]

I am trying to figure out how to add new indicators to ta-lib. I have copied the .c file in the ta_func folder and also added a header for the function in the ta_func.h file in the include folder. I have then compiled the library. I see the .c file in the right directory however I do not see the .o or .lo file. Additionally, python is unable to see my function.

Any help would be much appreciated.

[hhashim1]

Ok so I did a rebuild on the gen_code solution and then ran the file on my local machine. I then ran the git commands you mentioned above git add ... and git commit .... What should I do next?

[trufanov-nok]

launch docker, when test if sourcecode is buildable on windows after that

[hhashim1]

still cant access TSV :'(

[trufanov-nok]

And you shouldn't be able to access it until you implement a wrapper function for it in python wrapper. Did you do that?

[hhashim1]

What do you mean? I ran setup.py for ta-lib-wrapper. Is that what you mean?

[hhashim1]

Do I need to rebuild the whole solution in Visual Studio?

[trufanov-nok]

I asked you to replace pip install ta-lib with python setup.py install of a local copy of wrapper sourcecodes, because you will need to add a wrapper for C-library's TSV function in it. @mrjbq7 provide you with a hints on how one can add a wrapper function for a new C function: https://github.com/mrjbq7/ta-lib/issues/515#issuecomment-1115202637

Have you done this?

[hhashim1]

Yes, I ran python setup.py install but I did not run the commands which @mrjbq7 shared. I will do that now.

[mrjbq7]

I appreciate both of you working to make this easier -- if it makes sense afterward maybe we can have this as a document / README somewhere...

[hhashim1]

@mrjbq7 I agree. This has been a good process and I think we should document it. Going through the process multiple times and having hiccups in the process has taught me a valuable lesson. I will help in putting together a document for this process. Appreciate both of yall's help on this.

[hhashim1]

@trufanov-nok and @mrjbq7 I have gone through the series of commands shared by @mrjbq7 however I am still getting the same error AttributeError: module 'talib' has no attribute 'TSV'.

@trufanov-nok I was looking at ta_func.h is TSV was now listed in this file by gen_code and it is there now. Not sure what could've gone wrong.

[hhashim1]

For some reason, my logic in TSV was overwritten and replaced with the template code that I previously had. It must've happened during gen_code or one of the processes. At this point, I am thinking about deleting everything and restarting it tomorrow.

I will keep you updated on the status. This will also help me put together the steps for the document and then we can refine it later.

[mrjbq7]

Sooooo, short version, try this:

>>> import talib._ta_lib
>>> talib._ta_lib.TSV

Long version, you may need to add "TSV" to the list of __TA_FUNCTION_NAMES__ at the end of talib/_func.pxi.

That is a list of all function names to export and is updated automatically by my tools/generate_func.py wrapper.

[trufanov-nok]

For some reason, my logic in TSV was overwritten and replaced with the template code that I previously had. It must've happened during gen_code or one of the processes.

Note that gen_code overwrites everything inside it's "sections". For example everything between: /**** START GENCODE SECTION 2 - DO NOT DELETE THIS LINE ****/ and /**** END GENCODE SECTION 2 - DO NOT DELETE THIS LINE ****/

After initial replacement of %%%GENCODE%%% keywords it uses these comments to detect the pieces of code it may overwrite with generated function headers and default checks for arguments. If you need to modify something inside (like add a new optional argument to indicator) - you need to modify function declaration in table_t.c and launch gen_code to let it add it for you.

[hhashim1]

@mrjbq7 I get an error when importing. What does this tell you?

[mrjbq7]

import talib._ta_lib

On Wed, May 4, 2022 at 6:32 AM hhashim1 @.***> wrote:

@mrjbq7 https://github.com/mrjbq7 I get an error when importing. What does this tell you?

[image: image] https://user-images.githubusercontent.com/62855649/166691701-fc32639f-09c6-4cec-b63b-3f924aa3ec92.png

— Reply to this email directly, view it on GitHub https://github.com/mrjbq7/ta-lib/issues/515#issuecomment-1117319027, or unsubscribe https://github.com/notifications/unsubscribe-auth/AAAF5A6F6ZIDVJTOTFMDLQTVIJ35XANCNFSM5UNUHCSA . You are receiving this because you were mentioned.Message ID: @.***>

[hhashim1]

ah! I missed that first underscore. Still no good.

TSV is in the _func.pxi file

@boundscheck(False) # turn off bounds-checking for entire function
def TSV( np.ndarray close not None ):
    """ TSV(close)

    TSV

    Inputs:
        prices: ['close']
    Outputs:
        real
    """
    cdef:
        np.npy_intp length
        int begidx, endidx, lookback
        TA_RetCode retCode
        int outbegidx
        int outnbelement
        np.ndarray outreal
    close = check_array(close)
    length = close.shape[0]
    begidx = check_begidx1(length, <double*>(close.data))
    endidx = <int>length - begidx - 1
    lookback = begidx + lib.TA_TSV_Lookback( )
    outreal = make_double_array(length, lookback)
    retCode = lib.TA_TSV( 0 , endidx , <double *>(close.data)+begidx, &outbegidx , &outnbelement , <double *>(outreal.data)+lookback )
    _ta_check_success("TA_TSV", retCode)
    return outreal```

[mrjbq7]

And you ran “make cython” to update the C files before building?

On Wed, May 4, 2022 at 6:40 AM hhashim1 @.***> wrote:

ah! I missed that first underscore. Still no good.

[image: image] https://user-images.githubusercontent.com/62855649/166692681-e0e3df42-d2e2-4ba2-b799-20578ebb6444.png

TSV is in the _func.pxi file

@boundscheck(False) # turn off bounds-checking for entire function def TSV( np.ndarray close not None ): """ TSV(close)

TSV

Inputs:
    prices: ['close']
Outputs:
    real
"""
cdef:
    np.npy_intp length
    int begidx, endidx, lookback
    TA_RetCode retCode
    int outbegidx
    int outnbelement
    np.ndarray outreal
close = check_array(close)
length = close.shape[0]
begidx = check_begidx1(length, <double*>(close.data))
endidx = <int>length - begidx - 1
lookback = begidx + lib.TA_TSV_Lookback( )
outreal = make_double_array(length, lookback)
retCode = lib.TA_TSV( 0 , endidx , <double *>(close.data)+begidx, &outbegidx , &outnbelement , <double *>(outreal.data)+lookback )
_ta_check_success("TA_TSV", retCode)
return outreal```

— Reply to this email directly, view it on GitHub https://github.com/mrjbq7/ta-lib/issues/515#issuecomment-1117328065, or unsubscribe https://github.com/notifications/unsubscribe-auth/AAAF5A3NOSVDAXWIJUOFZ7LVIJ45TANCNFSM5UNUHCSA . You are receiving this because you were mentioned.Message ID: @.***>

[hhashim1]

Yes. However, there were some errors that I seemed to have ignored.

[trufanov-nok]

cython: Command not found Try to install cython3 package with sudo pip install cython or sudo apt install cython3

[hhashim1]

I installed cython3 however I am still getting the error.

[trufanov-nok]

try sudo ln -s /usr/bin/cython3 /usr/local/bin/cython after sudo apt install cython3

[mrjbq7]

I don’t know about cython3 package but does it rename the binary to cython3 to prevent shadowing the python2 version?

On May 4, 2022, at 8:10 AM, hhashim1 @.***> wrote:

 I installed cython3 however I am still getting the error.

— Reply to this email directly, view it on GitHub, or unsubscribe. You are receiving this because you were mentioned.

[hhashim1]

@trufanov-nok your suggestion worked however I got lots of errors when I did make cython

Seems like I got multiple errors for each indicator.

[mrjbq7]

You need to make sure the ta lib C library header files are in a known location.

Provide them TA_LIBRARY_PATH and TA_INCLUDE_PATH or install in standard location.

On May 4, 2022, at 8:33 AM, hhashim1 @.***> wrote:

 @trufanov-nok your suggestion worked however I got lots of errors when I did make cython

Seems like I got multiple errors for each indicator.

— Reply to this email directly, view it on GitHub, or unsubscribe. You are receiving this because you were mentioned.

[trufanov-nok]

@mrjbq7 Hmm. You're right, it seems cython3 package is some kind of supplementary. Perhaps for sudo pip install cython

I just searched my Kubuntu repository for cython with sudo apt-cache search cython and the only relevant package there is cython3 - C-Extensions for Python 3 which contains only a few files:

$ dpkg -S cython3
cython3: /usr/share/doc/cython3/changelog.Debian.gz
cython3: /usr/share/doc/cython3/README.Debian
cython3: /usr/bin/cython3
cython3-dbg: /usr/share/doc/cython3-dbg
cython3: /usr/share/doc/cython3
cython3: /usr/share/doc/cython3/copyright
cython3: /usr/share/doc/cython3/upstream/metadata
cython3: /usr/share/doc/cython3/upstream

And /usr/bin/cython3 is basically a small python3 script:

#!/usr/bin/python3

#
#   Cython -- Main Program, Unix
#

from Cython.Compiler.Main import main
main(command_line = 1)

[hhashim1]

cython and cython3 both give me the same error. I went ahead and remove both of the libraries and then reinstalled cython but still the same error.

[trufanov-nok]

@hhashim1 It's not about cython now. It's about wrapper code can't find TA-Lib's C code.

You must build and successfully build and install ta-lib's C sourcecode before building the wrapper:

./configure --prefix=/usr and then make
make
make install

make install will install library and headers to /usr/lib/ and /usr/include/ta-lib/. These are folders where wrapper can easily find them without setting TA_LIBRARY_PATH and TA_INCLUDE_PATH.

[trufanov-nok]

Execute ls -hs /usr/include/ta-lib/ta_func.h to check if this file exists before compiling a wrapper

[hhashim1]

@trufanov-nok running make gave me this error

[trufanov-nok]

Have you forgot about this: https://github.com/mrjbq7/ta-lib/issues/515#issuecomment-1114035127 automake is a part of automake package.

[hhashim1]

@trufanov-nok that worked and I have added that step to my list but I am still having other issues. I have decided to redownload everything and redo everything again. I cannot find the git command to be used to download the original repository that includes all the languages. I cant recall how I downloaded it last time. Do you recall?

[hhashim1]

@trufanov-nok how are you using your RT repo with the wrapper? Earlier you had mentioned that your repo has new functions that the wrapper repo is not aware of. Do you have an update of some sort to update the wrapper with your new functions?

[hhashim1]

@trufanov-nok I found the link...I thought this was for your RT repo but its actually your backup git clone -b original https://github.com/trufanov-nok/ta-lib-rt.git ta-lib

[trufanov-nok]

@hhashim1

how are you using your RT repo with the wrapper? Earlier you had mentioned that your repo has new functions that the wrapper repo is not aware of. Do you have an update of some sort to update the wrapper with your new functions?

I forked this wrapper and adapted it for using with a new library and its new functions: https://github.com/trufanov-nok/ta-lib-py-wrapper It won't help you as it contains a lot of changes and it's experimental. It's just a proof of concept.

[trufanov-nok]

Hmm... PVT indicator doesn't exists in original TA-Lib. It's an example indicator that I'm adding in my article about new indicator addition: https://github.com/trufanov-nok/ta-lib-rt/wiki/Adding-new-TA-function
Perhaps you've copied some code from this article and forget to rename PVT to TSV?

[hhashim1]

I am going through the steps to test it with PVT to make sure things are working before I use my code for TSV. So far it's working out well and I've taken good notes. :-)

[hhashim1]

@trufanov-nok I am stuck on the end of your instruction page. I am not sure where I need to add the below code. Which file and at what location should I add the below code?

VALUE_HANDLE_DEREF(outReal) = STATE.prevPVT;

PUSH_TO_MEM(inClose,inClose); // not pushing inValue as not need it.
return ENUM_VALUE(RetCode,TA_SUCCESS,Success);```

What is the `In main state function`?

[trufanov-nok]

@hhashim1 state functions don't exist in original ta-lib. That's for my fork. Ignore these paragraphs.

[hhashim1]

@mrjbq7 I was able to compile ta-lib with no errors and then went on to follow your instructions on compiling ta-lib-wrapper. I still cannot see the test indicator in python. Here is the screenshot.

I did have some errors during make build and make test. Here is the screenshot of the beginning of make build.

Can you tell me what you think might have gone wrong here?

Here is the end of make build, the beginning of make test and make install

[trufanov-nok]

Everything looks fine.. These are warnings, not errors. This kind of warnings can be ignored.

As for make test - it can't find a pytest - it can be ignored too. It seems make tries to call python's pytest module from a command line like it tried to do with cython before. I guess that require sudo pip install pytest then sudo apt install python3-pytest to get a wrapper for it (/usr/bin/pytest-3) then sudo ln -s /usr/bin/pytest-3 /usr/local/bin/pytest to make it callable from command line for make test.
But as I said - it may be ignored.

I would like to see how you added a PVT wrapper to ta-lib-wrapper project.
You can call git diff to display all uncommited changes you made in this project. Or git show if they're commited.

[hhashim1]

BTW, I ran the whole ta-lib-wrapper process again and then ran python setup.py install. I got an error for that. What does this mean? @trufanov-nok @mrjbq7

[hhashim1]

Everything looks fine.. These are warnings, not errors. This kind of warnings can be ignored.

As for make test - it can't find a pytest - it can be ignored too. It seems make tries to call python's pytest module from a command line like it tried to do with cython before. I guess that require sudo pip install pytest then sudo apt install python3-pytest to get a wrapper for it (/usr/bin/pytest-3) then sudo ln -s /usr/bin/pytest-3 /usr/local/bin/pytest to make it callable from command line for make test. But as I said - it may be ignored.

I would like to see how you added a PVT wrapper to ta-lib-wrapper project. You can call git diff to display all uncommited changes you made in this project. Or git show if they're commited.

here is git show

[trufanov-nok]

and git diff?

then ran python setup.py install. I got an error for that

what's error text?

[hhashim1]

Here is the updated screenshot

[trufanov-nok]

this is called from ta-lib folder which is a C library sourcebase. I would like to see a git diff called from ta-lib-wrapper folder, which is a python wrapper sourcebase

[hhashim1]

its got lots of stuff...

[trufanov-nok]

Save it to log.txt with git diff > log.txt and upload somewhere or paste the content on pastebin.com with 2 weeks expiration date.

[trufanov-nok]

or you may zip the log.txt and upload to this discussion thread

[hhashim1]

log.zip

Here you go @trufanov-nok