[BUG] Install via Source code (tar.gz) failed

axot commented 3 years ago

We had been installed ddtrace via [1] before, in the recent version I got an error here,

So now I have been tried to use [2] for the installation, but there is no composer.json file in [2] which we need for composer installation in php application side.

If pecl's and PHP application's ddtrace version needn't be a same version, use pecl for installation will be an easy way. Otherwise, we need some workaround to make it works.

For a temporary workaround, I added make generate when build extension using [1].

[1]. Source code (tar.gz) [2]. datadog-php-tracer-0.50.0.x86_64.tar.gz

labbati commented 3 years ago

Hi @axot!

If you cannot use the .deb/.rpm/.apk packages then [2] is the recommended way to install the extension.

Is you are not doing any manual instrumentation and you are only using our out of the box automatic instrumentation (which covers 99% of use cases) you are good to go, no composer.json required.

Is you are doing manual instrumentation and you need to add the composer dependency, then just add the same version as your installed version. In your case add to "require" the line "datadog/dd-trace": "0.50.0".

We recommend the extension and composer version to be same, as otherwise the output could be not easy to predict.

axot commented 3 years ago

Hi,

The problem we have is the base php image which include all pecl extensions was built by infra team, and the requirement of ddtrace in composer.json, a symbolic link path to the same source of pecl extensions', was written by application team. It is difficult to match the 2 versions because they are managed by different teams if requirng with "datadog/dd-trace": "0.50.0" in composer.json.

Normally, just in my experience, most pecl extensions does not need the exact same version used in composer.json. With a loose coupling pecl extension, It makes more easier to update library in composer side since this repo is actively in development.

labbati commented 3 years ago

Thank you for clarifying @axot.

In the future, which is where we are going with already planned work for Q1, the composer package will just be a shim providing an API to the underlying extension where versions will be able to be out of sync within certain boundaries.

At the moment we suggest to keep the two versions in sync to reduce the possibility of errors, even if we do not currently plan any breaking change and we will announce big changes to the API well ahead of time.

Just to iterate over my previous question: are you doing manual instrumentation? Why you do need to install the composer package? We might be able to provide a workaround there, instead.

axot commented 3 years ago

Hi @labbati,

Yes, until now, we have installed ddtrace by compiling source code from [1], we can't use prebuilt .deb/.rpm/.apk packages because we have been using custom-build php, almost same as php official's.

Do you mean if we install ddtrace by pecl or by [2] then it does not need to require ddtrace in composer.json? We could have a try tomorrow.

labbati commented 3 years ago

Do you mean if we install ddtrace by pecl or by [2] then it does not need to require ddtrace in composer.json? We could have a try tomorrow.

Correct. You need to add the composer dependency ONLY if in your php code you are doing some manual instrumentation and you are explicitly referring to classes/functions in the DDTrace\* namespace. If you are not, then you SHOULD NOT add a dependency to your composer file. You should instead do only either thepecl installation or the datadog-php-tracer-0.50.0.x86_64.tar.gz installation.

If you take the pecl approach: installation is pretty standard.

If you take the datadog-php-tracer-0.50.0.x86_64.tar.gzapproach: it does not make any assumption nor requires any specific dependency on any native php package. See installation instructions here. Also in case you want to install the tracer to the non default php binary (e.g. if instead of the php binary pointed by php command you want to install it to the php binary at /my/custom/location/bin/php which is not in the path, you can export DD_TRACE_PHP_BIN=/my/custom/location/bin/php before launching the installation post-install.sh that is referenced in the docs I linked above.

axot commented 3 years ago

Thanks, let me confirm tomorrow, I’m not so sure, we may used DDTrace* namespace for adding custom span to trace.

labbati commented 3 years ago

Hi @axot, did you have a chance to check is my last comment solves your issue?

axot commented 3 years ago

@labbati We have been heavily used custom span traces which need DDTrace* namespace. So I think we need use datadog-php-tracer-0.50.0.x86_64.tar.gz approach, for now we are still using the following code. Let me try to check the suggestion way you provide, and give you response in next week.

curl -sqLo - https://github.com/DataDog/dd-trace-php/archive/${DATADOG}.tar.gz | tar zx -C / \
&& cd /dd-trace-php-${DATADOG} \
&& make generate \
&& phpize \
&& ./configure --enable-ddtrace \
&& make -j $(nproc) && make install && make clean

axot commented 3 years ago

I tried the installation instructions,

With the first AUTOMATIC INI FILE SETUP method, I got the following errors,


$ cat /etc/os-release
NAME="Alpine Linux"
ID=alpine
VERSION_ID=3.12.1
PRETTY_NAME="Alpine Linux v3.12"
HOME_URL="https://alpinelinux.org/"
BUG_REPORT_URL="https://bugs.alpinelinux.org/"

$ curl -L -O https://github.com/DataDog/dd-trace-php/releases/download/0.54.0/datadog-php-tracer-0.54.0.x86_64.tar.gz

$ tar zxvf datadog-php-tracer-0.54.0.x86_64.tar.gz -C /

$ /opt/datadog-php/bin/post-install.sh

PHP version

PHP 7.4.12 (cli) (built: Nov 5 2020 20:50:00) ( NTS ) Copyright (c) The PHP Group Zend Engine v3.4.0, Copyright (c) Zend Technologies with Zend OPcache v7.4.12, Copyright (c), by Zend Technologies

Installing Datadog PHP tracing extension (ddtrace)

Logging /usr/local/bin/php -i to a file

Creating ddtrace.ini

; DO NOT EDIT THIS FILE ; To overwrite the INI settings for this extension, edit ; the INI file in this directory called "ddtrace-custom.ini"

[datadog] extension=/opt/datadog-php/extensions/ddtrace-20190902-alpine.so ddtrace.request_init_hook=/opt/datadog-php/dd-trace-sources/bridge/dd_wrap_autoloader.php

ddtrace.ini created

Linking ddtrace.ini for supported SAPI's

Found SAPI config directory: /usr/local/etc/php/conf.d

Linking ddtrace.ini to /usr/local/etc/php/conf.d/98-ddtrace.ini

Linking ddtrace-custom.ini to /usr/local/etc/php/conf.d/99-ddtrace-custom.ini

PHP Warning: PHP Startup: Unable to load dynamic library '/opt/datadog-php/extensions/ddtrace-20190902-alpine.so' (tried: /opt/datadog-php/extensions/ddtrace-20190902-alpine.so (Error loading shared library libexecinfo.so.1: No such file or directory (needed by /opt/datadog-php/extensions/ddtrace-20190902-alpine.so)), /usr/local/lib/php/extensions/no-debug-non-zts-20190902//opt/datadog-php/extensions/ddtrace-20190902-alpine.so.so (Error loading shared library /usr/local/lib/php/extensions/no-debug-non-zts-20190902//opt/datadog-php/extensions/ddtrace-20190902-alpine.so.so: No such file or directory)) in Unknown on line 0

Failed enabling ddtrace extension

The extension has been installed but couldn't be enabled

Try adding the extension manually to your PHP - php.ini - configuration file

e.g. by adding following line:

extension=/opt/datadog-php/extensions/ddtrace-20190902-alpine.so

Note that your PHP API version must match the extension's API version

PHP API version can be found using following command

/usr/local/bin/php -i | grep 'PHP API'

bash-5.0# ldd /opt/datadog-php/extensions/ddtrace-20190902-alpine.so /lib/ld-musl-x86_64.so.1 (0x7f5de1919000) libcurl.so.4 => /usr/lib/libcurl.so.4 (0x7f5de166b000) Error loading shared library libexecinfo.so.1: No such file or directory (needed by /opt/datadog-php/extensions/ddtrace-20190902-alpine.so) libc.musl-x86_64.so.1 => /lib/ld-musl-x86_64.so.1 (0x7f5de1919000) libnghttp2.so.14 => /usr/lib/libnghttp2.so.14 (0x7f5de1646000) libssl.so.1.1 => /lib/libssl.so.1.1 (0x7f5de15c5000) libcrypto.so.1.1 => /lib/libcrypto.so.1.1 (0x7f5de1345000) libz.so.1 => /lib/libz.so.1 (0x7f5de132b000)



2. With the Install from source method, I still need run `make generate` command, otherwise these is no a `_generated.php` file.

labbati commented 3 years ago

Hi @axot and thanks for getting back so quickly and let me try to wrap up all the options here.

Installation via .tar.gz You need to manually install apk add bash curl libexecinfo that we declare in our dependencies list

Installation via .apk You should now be able to use it as we dropped the dependency on php during installation (that I believe was what blocked you).

Installation via source.tar.gz (not recommended) You NEED to generate the _generated.php on your own as you correctly do.

Please let me know if we my latest recommendation you are able to install our tracer without source code.

axot commented 3 years ago

@labbati Thank you for the information, I success to install ddtrace-20190902-alpine.so via the APK method, and the result of php --ri=ddtrace shows it works, the next step we want php composer packaged could be installed the exact same version as the one installed by APK.

I tried to fix the path in repositories section in our project side of composer.json, this method worked when we installed ddtrace from source code. But now, there is no composer.json file in /opt/datadog-php/dd-trace-sources/, so this way not works again. The purpose here is we want to make sure both pecl binary and composer package of ddtrace has the same version.

{
    "require": {
        "datadog/dd-trace": "@dev",
    },
    "repositories": [
        { "type": "path", "url": "/opt/datadog-php/dd-trace-sources/", "options": { "symlink": false } },
    ]
}

Additional information,

# php --ri=ddtrace

ddtrace

Datadog PHP tracer extension
For help, check out the documentation at https://docs.datadoghq.com/tracing/languages/php/
(c) Datadog 2020

Datadog tracing support => enabled
Version => 0.54.0
DATADOG TRACER CONFIGURATION => {"date":"2021-01-28T01:31:02Z","os_name":"Linux 70a818cf4d9a 4.19.121-linuxkit #1 SMP Tue Dec 1 17:50:32 UTC 2020 x86_64","os_version":"4.19.121-linuxkit","version":"0.54.0","lang":"php","lang_version":"7.4.11","env":null,"enabled":true,"service":null,"enabled_cli":false,"agent_url":"http://localhost:8126","debug":false,"analytics_enabled":false,"sample_rate":1.000000,"sampling_rules":null,"tags":null,"service_mapping":null,"distributed_tracing_enabled":true,"priority_sampling_enabled":true,"dd_version":null,"architecture":"x86_64","sapi":"cli","ddtrace.request_init_hook":"/opt/datadog-php/dd-trace-sources/bridge/dd_wrap_autoloader.php","open_basedir_configured":false,"uri_fragment_regex":null,"uri_mapping_incoming":null,"uri_mapping_outgoing":null,"auto_flush_enabled":false,"generate_root_span":true,"http_client_split_by_domain":false,"measure_compile_time":true,"report_hostname_on_root_span":false,"traced_internal_functions":null,"auto_prepend_file_configured":false,"integrations_disabled":null,"enabled_from_env":true,"opcache.file_cache":null,"agent_error":"Failed to connect to localhost port 8126: Connection refused"}

                               Diagnostics
agent_error => Failed to connect to localhost port 8126: Connection refused
Diagnostic checks => failed

Directive => Local Value => Master Value
ddtrace.disable => Off => Off
ddtrace.request_init_hook => /opt/datadog-php/dd-trace-sources/bridge/dd_wrap_autoloader.php => /opt/datadog-php/dd-trace-sources/bridge/dd_wrap_autoloader.php

labbati commented 3 years ago

Hi @axot let me share some thoughts.

First of all we see the pain of doing this and we started an internal conversation (putting together experiences from you and other users' use cases) to provide a more pleasant experience while doing this.

In the meantime I wanted to share a few thoughts.

I think the best way for you to achieve what you want to achieve is to use a psr-4 namespace definition rather than defining an additional repository. So something like (remember to composer install after you edit the file)

{
    .....
    "autoload": {
        "psr-4": {
        .........   
            "DDTrace\\": "/opt/datadog-php/dd-trace-sources/src/api/"
    }
    }
    .....
}

This would work for Linux and MAC users, but will not for windows who don't have /opt/.... So basically with this approach only Linux+MAC users OR Windows users, not both.

However, another possibility you should consider is to keep the two versions slightly out of sync. We do not plan to apply breaking changes to the composer packages for the near future. So a process that can work for you is:

fix the composer package to the latest 0.54.0
install on your servers the extension 0.54.0
when you update either the package or the extension, accept that they are out of sync a few versions. Composer package is just a Noop, so what will really matter is the extension installed.
we will open a deprecation note issue well in advance if we will have to break compatibility at a certain point.

axot commented 3 years ago

Hi @labbati,

I discussed this internally, use psr-4 autoload is a smart way, one concern of this method is, We have to install the same version of ddtrace in Mac, because our local development environment is using docker-composer, it is complicated to setup PHP IDE(eg: phpstorm) reference files located at /opt/datadog-php/dd-trace-sources/src/api/ which inside of container.

The method of keeping the two versions slightly out of sync, we have experienced breaking changes behavior before, that make us to avert this way to manage different version of ddtrace.

labbati commented 3 years ago

Hi @axot

The method of keeping the two versions slightly out of sync, we have experienced breaking changes behavior before, that make us to avert this way to manage different version of ddtrace.

Did it happen after or before 0.46.0? After we merged https://github.com/DataDog/dd-trace-php/pull/877 then our src/api classes (the only ones you should ever use) are stable.

axot commented 3 years ago

Hi, it happened once we had been upgrading from 0.44.1 to 0.46.0.

labbati commented 3 years ago

That makes sense. but after 0.46.0 we expect not to introduce any breaking changes to the files you are supposed to use (src/api)

axot commented 3 years ago

Hi @labbati, is there some way we can make sure that there will be no break changes in the following versions. For an instance, if there is a rule that make sure all break changes will and only will happens in major version changes.

labbati commented 3 years ago

Not in a structural way, other than we commit not to introduced breaking changes in classes from src/api. If we will be forced to introduce them, we will open issues before and add a note the the release as a breaking change.

I think that for what concerns the original issue this thread can be considered closed. Let me know if you feel like more work has to be done and I will reopen it.

Thanks for all your feedback

DataDog / dd-trace-php