juanlofer-eprosima
commented
2 years ago Thanks @lurenlym for such an in-depth report!
Reviewing your modifications I have noticed some lines of code were not transferred between publisher and subscriber in the exchange of TCP roles. Would you please test the following configuration and see if it makes any difference? That would be very valuable for us before further looking into this issue.
The main modified code of Publisher init:
DomainParticipantQos pqos;
std::shared_ptr<TCPv4TransportDescriptor> descriptor =std::make_shared<TCPv4TransportDescriptor>();
pqos.wire_protocol().builtin.discovery_config.leaseDuration =eprosima::fastrtps::c_TimeInfinite;
pqos.wire_protocol().builtin.discovery_config
leaseDuration_announcementperiod = eprosima::fastrtps::Duration_t(5, 0);
int32_t kind = LOCATOR_KIND_TCPv4;
Locator initial_peer_locator;
initial_peer_locator.kind = kind;
for (std::string ip : whitelist)
{
descriptor->interfaceWhiteList.push_back(ip);
std::cout << "Whitelisted " << ip << std::endl;
}
if (!wan_ip.empty())
{
IPLocator::setIPv4(initial_peer_locator, wan_ip);
std::cout << wan_ip << ":" << port << std::endl;
}
else
{
IPLocator::setIPv4(initial_peer_locator, "127.0.0.1");
}
initial_peer_locator.port = port;
pqos.wire_protocol().builtin.initialPeersList.push_ba
(initial_peer_locator); // Subscriber's meta channel
pqos.name("client");
pqos.transport().use_builtin_transports = false;
pqos.transport().user_transports.push_back(descriptor);
participant_ = DomainParticipantFactory::get_instanc
(->create_participant(0, pqos);
if (participant_ == nullptr)
{
return false;
}
//REGISTER THE TYPE
type_.register_type(participant_);
//CREATE THE PUBLISHER
publisher_ = participant_->create_publisher(PUBLISHER_QOS_DEFAULT);
if (publisher_ == nullptr)
{
return false;
}
//CREATE THE TOPIC
topic_ = participant_->create_topic("HelloWorldTopicTCP", "HelloWorld", TOPIC_QOS_DEFAULT);
if (topic_ == nullptr)
{
return false;
}
//CREATE THE DATAWRITER
DataWriterQos wqos;
wqos.history().kind = KEEP_LAST_HISTORY_QOS;
wqos.history().depth = 1;
wqos.resource_limits().max_samples = 50;
wqos.resource_limits().allocated_samples = 20;
wqos.reliable_writer_qos().times.heartbeatPeriod.seconds = 2;
wqos.reliable_writer_qos().times.heartbeatPeriod.nanosec = 200 * 1000 * 1000;
wqos.reliability().kind = RELIABLE_RELIABILITY_QOS;
writer_ = publisher_->create_datawriter(topic_, wqos, &listener_);
if (writer_ == nullptr)
{
return false;
}
return true;The main modified code of Subscriber init:
//CREATE THE PARTICIPANT
DomainParticipantQos pqos;
std::shared_ptr<TCPv4TransportDescriptor> descriptor =
std::make_shared<TCPv4TransportDescriptor>();
pqos.wire_protocol().builtin.discovery_config.leaseDuration = eprosima::fastrtps::c_TimeInfinite;
pqos.wire_protocol().builtin.discovery_config.leaseDuration_announcementperiod = eprosima::fastrtps::Duration_t(5, 0);
for (std::string ip : whitelist)
{
descriptor->interfaceWhiteList.push_back(ip);
std::cout << "Whitelisted " << ip << std::endl;
}
if (!wan_ip.empty())
{
descriptor->set_WAN_address(wan_ip);
std::cout << wan_ip << ":" << port << std::endl;
}
descriptor->add_listener_port(port);
pqos.name("server");
if (use_tls)
{
using TLSVerifyMode
=TCPTransportDescriptor::TLSConfig::TLSVerifyMode;
using TLSOptions =TCPTransportDescriptor::TLSConfig::TLSOptions;
descriptor->apply_security = true;
descriptor->tls_config.verify_file = "cacert.pem";
descriptor->tls_config.verify_mode =TLSVerifyMode::VERIFY_PEER;
descriptor->tls_config.add_optio(TLSOptions::DEFAULT_WORKAROUNDS);
}
pqos.transport().use_builtin_transports = false;
pqos.transport().user_transports.push_back(descriptor);
participant_ = DomainParticipantFactory::get_instance()->create_participant(0, pqos);
if (participant_ == nullptr)
{
return false;
}
//REGISTER THE TYPE
type_.register_type(participant_);
// //CREATE THE SUBSCRIBER
subscriber_ = participant_->create_subscriber(SUBSCRIBER_QOS_DEFAULT);
if (subscriber_ == nullptr)
{
return false;
}
//CREATE THE TOPIC
topic_ = participant_->create_topic("HelloWorldTopicTCP", "HelloWorld", TOPIC_QOS_DEFAULT);
if (topic_ == nullptr)
{
return false;
}
//CREATE THE DATAREADER
DataReaderQos rqos;
rqos.history().kind = eprosima::fastdds::dds::KEEP_LAST_HISTORY_QOS;
rqos.history().depth = 1;
rqos.resource_limits().max_samples = 50;
rqos.resource_limits().allocated_samples = 20;
rqos.reliability().kind = eprosima::fastdds::dds::RELIABLE_RELIABILITY_QOS;
rqos.durability().kind = eprosima::fastdds::dds::TRANSIENT_LOCAL_DURABILITY_QOS;
reader_ = subscriber_->create_datareader(topic_, rqos, &listener_);
if (reader_ == nullptr)
{
return false;
}
return true;
lurenlym
JLBuenoLopez
Environment
version: master branch cloned on 10/20 2021 895197a9 os: ubuntu 18.04 arch: x86
Question
It is normal to use DDSHelloWorldExampleTCP demo to transmit 900k of data per 100ms. But when I exchanged the server and client in the code something is wrong. The transmission speed is significantly slower than before.
The main modified code of Publisher init:
The main modified code of Subscriber init:
publisher code
test designed and CLI
computer A ip 10.10.10.1 publisher computer B ip 10.3.1.102 subscriber In A:
In B:
Observe bandwidth usage
Publisher as server and Subscriber as client:(original code)
It seems the publisher as the client will transmit three to four times more data, then try to debug source code to find the most time-consuming part of the program. I found following code will bring different time consumption differences
src/cpp/rtps/participant/RTPSParticipantImpl.h
900k data will be divided into 16 segments(due to 64k limits) The results shows the code above will cost about 200us~1000us while publisher as server after client matched. But it seems that code flush twice in one frame.
I observed the call stack and found the "pulish(false)" with call function https://github.com/eProsima/Fast-DDS/blob/4a27fbbbeb08ad83ffa10422eeaa201017382905/src/cpp/rtps/writer/StatefulWriter.cpp#L2017 normally in and
https://github.com/eProsima/Fast-DDS/blob/7e617577da511d89ccdbb42293c795254d07b964/src/cpp/rtps/flowcontrol/FlowControllerImpl.hpp#L1146 the variable send_resourcelist size is 3.
When I exchange the server and client:
We can see that transmitting each seg data will take hundreds of times the time include the heartbeat frame. One of the biggest factors is the "send_resourcelist" size is 36. This seems to be caused by repeatedly converting the same address to this variable when establishing a TCP connection. I found that while server and client are not matched, the "send_resourcelist" is 1 normally. But when the server is open, "send_resourcelist" will be added abnormally.
Q1: Is it a normal phenomenon in test1(publisher as a server)? It looks like I should change the relevant parameters of flowcontrol
Q2: How to accelerate while server as a client? It seems that the createSenderResources function is called multiple times, is it a bug? In a normal peer-to-peer connection case, How much should this variable be, and what factors are related?