[BUG]: CrossEncoder.predict() with num_workers > 0 returns constants

[hkristof03]

Hi,

The usage of the CrossEncoder.predict() method with num_workers > 0 returns constant values. The below example returns full zeros. I haven't tested it on Linux, but this never occurred to me in PyTorch or Tensorflow dataloaders with multiprocessing based data loading. Moreover, the usage of multiple workers reduced the speed of the inference.

from sentence_transformers import CrossEncoder

questions_chunks = [
    ('When did the torch arrive in Canberra?',
    "Australia: The event was held in Canberra, Australian Capital Territory on April 24, and covered around 16 km of Canberra's central areas, from Reconciliation Place to Commonwealth Park. Upon its arrival in Canberra, the Olympic flame was presented by Chinese officials to local Aboriginal elder Agnes Shea, of the Ngunnawal people. She, in turn, offered them a message stick, as a gift of peace and welcome. Hundreds of pro-Tibet protesters and thousands of Chinese students reportedly attended."),
    ('When did the torch arrive in Canberra?',
    'Amid pressure from both directions, Prime Minister Gordon Brown welcomed the torch outside 10 Downing Street without holding or touching it. The London relay saw the torch surrounded by what the BBC described as "a mobile protective ring." Protests began as soon as Redgrave started the event, leading to at least thirty-five arrests. In Ladbroke Grove a demonstrator attempted to snatch the torch from Konnie Huq in a momentary struggle, and in a separate incident, a fire extinguisher was set off near the torch.'),
    ('When did the torch arrive in Canberra?',
    'Following the events in Olympia, there were reports that China requested permission to deploy People\'s Liberation Army personnel along the relay route to protect the flame in Canberra. Australian authorities stated that such a request, if it were to be made, would be refused. Chinese officials labeled it a rumor. Australian police have been given powers to search relay spectators, following a call by the Chinese Students and Scholars Association for Chinese Australian students to "go defend our sacred torch" against "ethnic degenerate scum and anti-China separatists".'),
    ('When did the torch arrive in Canberra?',
    'But we have assurances that it will be done peacefully.". Also, Australia\'s ACT Chief Minister, Jon Stanhope confirmed that the Chinese embassy was closely involve to ensure that "pro-China demonstrators vastly outnumbered Tibetan activists." Australian freestyle swimmer and five-time Olympic gold medalist Ian Thorpe ended the Australian leg of the torch relay April 24, 2008, touching the flame to light a cauldron after a run that was only marginally marked by protests. People demonstrated both for China and for Tibet. At least five people were arrested during the torch relay.'),
    ('When did the torch arrive in Canberra?',
    'Vietnam: The event was held in Ho Chi Minh City on April 29. Some 60 torchbearers carried the torch from the downtown Opera House to the Military Zone 7 Competition Hall stadium near Tan Son Nhat International Airport along an undisclosed route. Vietnam is involved in a territorial dispute with China (and other countries) for sovereignty of the Spratly and Paracel Islands; tensions have risen recently[when?]'),
    ('When did the torch arrive in Canberra?',
    'Kazakhstan: The first torchbearer in Almaty, where the Olympic torch arrived for the first time ever on April 2, was the President of Kazakhstan Nursultan Nazarbaev. The route ran 20 km from Medeo stadium to Astana Square. There were reports that Uighur activists were arrested and some were deported back to China.'),
    ('When did the torch arrive in Canberra?',
    "Macao: The event was held in Macau on May 3. It was the first time that the Olympic torch had traveled to Macau. A ceremony was held at Macau Fisherman's Wharf. Afterward, the torch traveled through Macau, passing by a number of landmarks including A-Ma Temple, Macau Tower, Ponte Governador Nobre de Carvalho, Ponte de Sai Van, Macau Cultural Centre, Macau Stadium and then back to the Fisherman's Wharf for the closing ceremony."),
    ('When did the torch arrive in Canberra?',
    'After being lit at the birthplace of the Olympic Games in Olympia, Greece on March 24, the torch traveled to the Panathinaiko Stadium in Athens, and then to Beijing, arriving on March 31. From Beijing, the torch was following a route passing through six continents. The torch has visited cities along the Silk Road, symbolizing ancient links between China and the rest of the world. The relay also included an ascent with the flame to the top of Mount Everest on the border of Nepal and Tibet, China from the Chinese side, which was closed specially for the event.'),
    ('When did the torch arrive in Canberra?',
    'China: The torch returned to China for the first time since April. The torch arrived in Sanya, Hainan on May 4 with celebrations attended by International Olympic Committee (IOC) officials and Chinese big names like Jackie Chan. The entire relay through Mainland China was largely a success with many people welcoming the arrival of the torch along the way.'),
    ('When did the torch arrive in Canberra?',
    'Russia: On April 5 the Olympic torch arrived at Saint Petersburg, Russia. The length of the torch relay route in the city was 20 km, with the start at the Victory Square and finish at the Palace Square. Mixed martial arts icon and former PRIDE Heavyweight Champion Fedor Emelianenko was one the torch bearers. This gives him the distinction of the being the first active MMA fighter to carry the Olympic flame.'),
    ('Who received the flame from Chinese officials in Canberra?',
    "Australia: The event was held in Canberra, Australian Capital Territory on April 24, and covered around 16 km of Canberra's central areas, from Reconciliation Place to Commonwealth Park. Upon its arrival in Canberra, the Olympic flame was presented by Chinese officials to local Aboriginal elder Agnes Shea, of the Ngunnawal people. She, in turn, offered them a message stick, as a gift of peace and welcome. Hundreds of pro-Tibet protesters and thousands of Chinese students reportedly attended."),
    ('Who received the flame from Chinese officials in Canberra?',
    'Following the events in Olympia, there were reports that China requested permission to deploy People\'s Liberation Army personnel along the relay route to protect the flame in Canberra. Australian authorities stated that such a request, if it were to be made, would be refused. Chinese officials labeled it a rumor. Australian police have been given powers to search relay spectators, following a call by the Chinese Students and Scholars Association for Chinese Australian students to "go defend our sacred torch" against "ethnic degenerate scum and anti-China separatists".'),
    ('Who received the flame from Chinese officials in Canberra?',
    'Hundreds of pro-Tibet protesters and thousands of Chinese students reportedly attended. Demonstrators and counter-demonstrators were kept apart by the Australian Federal Police. Preparations for the event were marred by a disagreement over the role of the Chinese flame attendants, with Australian and Chinese officials arguing publicly over their function and prerogatives during a press conference.'),
    ('Who received the flame from Chinese officials in Canberra?',
    'But we have assurances that it will be done peacefully.". Also, Australia\'s ACT Chief Minister, Jon Stanhope confirmed that the Chinese embassy was closely involve to ensure that "pro-China demonstrators vastly outnumbered Tibetan activists." Australian freestyle swimmer and five-time Olympic gold medalist Ian Thorpe ended the Australian leg of the torch relay April 24, 2008, touching the flame to light a cauldron after a run that was only marginally marked by protests. People demonstrated both for China and for Tibet. At least five people were arrested during the torch relay.'),
    ('Who received the flame from Chinese officials in Canberra?',
    'Tony Goh, chairman of the Australian Council of Chinese Organisations, has said the ACCO would be taking "thousands" of pro-Beijing demonstrators to Canberra by bus, to support the torch relay. Zhang Rongan, a Chinese Australian student organising pro-Beijing demonstrations, told the press that Chinese diplomats were assisting with the organization of buses, meals and accommodation for pro-Beijing demonstrators, and helping them organise a "peaceful show of strength".'),
    ('Who received the flame from Chinese officials in Canberra?',
    'Up to 600 pro-Tibet protesters were expected to attend the relay, along with between 2,000 and 10,000 Chinese supporters. Taking note of the high number of Chinese supporters, Ted Quinlan, head of the Canberra torch relay committee, said: "We didn\'t expect this reaction from the Chinese community. It is obviously a well-coordinated plan to take the day by weight of numbers. But we have assurances that it will be done peacefully.".'),
    ('Who received the flame from Chinese officials in Canberra?',
    'This was later denied by the Chinese Ministry of Foreign Affairs, despite video footage broadcast by French television network France 2 which showed Chinese flame attendants extinguishing the torch. Backup flames are with the relay at all times to relight the torch. French judoka and torchbearer David Douillet expressed his annoyance at the Chinese flame attendants who extinguished the torch which he was about to hand over to Teddy Riner: "I understand they\'re afraid of everything, but this is just annoying.'),
    ('Who received the flame from Chinese officials in Canberra?',
    'Some Western media have reported on Chinese accusations of Western media bias. The Daily Telegraph published an opinion piece by the Chinese ambassador to the United Kingdom, Fu Ying, who accused Western media of "demonising" China during their coverage of the torch relays. The Telegraph also asked its readers to send their views in response to the question "Is the West demonising China?" The BBC reported on a demonstration in Sydney by Chinese Australians "voicing support for Beijing amid controversy over Tibet" and protesting against what they saw as Western media bias.'),
    ('Who received the flame from Chinese officials in Canberra?',
    'Indonesia: The Olympic flame reached Jakarta on April 22. The original 20 km relay through Jakarta was cancelled due to "security worries", at the request of the Chinese embassy, and the torch was instead carried round the city main\'s stadium, as it had been in Islamabad. Several dozen pro-Tibet protesters gathered near the stadium, and were dispersed by the police. The event was held in the streets around the city main\'s stadium. The cancelling of the relay through the city itself was decided due to security concerns and at the request of the Chinese embassy.'),
    ('Who received the flame from Chinese officials in Canberra?',
    'The Beijing Olympic Organizing Committee sent out a team of 30 unarmed attendants selected from the People\'s Armed Police to escort the flame throughout its journey. According to Asian Times, sworn in as the "Beijing Olympic Games Sacred Flame Protection Unit" during a ceremony in August 2007, their main job is to keep the Olympic flame alight throughout the journey and to assist in transferring the flame between the torches, the lanterns and the cauldrons. They wear matching blue tracksuits and are intended to accompany the torch every step of the way.'),
    ('What did Agnes Shea give to the Chinese in return?',
    'The city of Beijing was rebuilt with new palace grounds that included artificial lakes, hills and mountains, and parks. During the Yuan period, Beijing became the terminus of the Grand Canal of China, which was completely renovated. These commercially oriented improvements encouraged overland and maritime commerce throughout Asia and facilitated direct Chinese contacts with Europe. Chinese travelers to the West were able to provide assistance in such areas as hydraulic engineering. Contacts with the West also brought the introduction to China of a major food crop, sorghum, along with other foreign food products and methods of preparation.'),
    ('What did Agnes Shea give to the Chinese in return?',
    'According to Tibetologist John Powers, Tibetan sources counter this narrative of titles granted by the Chinese to Tibetans with various titles which the Tibetans gave to the Chinese emperors and their officials. Tribute missions from Tibetan monasteries to the Chinese court brought back not only titles, but large, commercially valuable gifts which could subsequently be sold. The Ming emperors sent invitations to ruling lamas, but the lamas sent subordinates rather than coming themselves, and no Tibetan ruler ever explicitly accepted the role of being a vassal of the Ming.'),
    ('What did Agnes Shea give to the Chinese in return?',
    'For the Han Chinese, however, it was a humiliating reminder of Qing authority that challenged traditional Confucian values. The Classic of Filial Piety (Xiaojing) held that "a person\'s body and hair, being gifts from one\'s parents, are not to be damaged." Under the Ming dynasty, adult men did not cut their hair but instead wore it in the form of a top-knot. The order triggered strong resistance to Qing rule in Jiangnan and massive killing of ethnic Han Chinese.'),
    ('What did Agnes Shea give to the Chinese in return?',
    'The Qing court sent forces to defeat Yaqub Beg and Xinjiang was reconquered, and then the political system of China proper was formally applied onto Xinjiang. The Kumul Khanate, which was incorporated into the Qing empire as a vassal after helping Qing defeat the Zunghars in 1757, maintained its status after Xinjiang turned into a province through the end of the dynasty in the Xinhai Revolution up until 1930. In early 20th century, Britain sent an expedition force to Tibet and forced Tibetans to sign a treaty.'),
    ('What did Agnes Shea give to the Chinese in return?',
    "Peter C. Perdue says that Wang Anshi (1021–1086), realizing that China could not produce enough militarily capable steeds, had also aimed to obtain horses from Inner Asia in exchange for Chinese tea. The Chinese needed horses not only for cavalry but also as draft animals for the army's supply wagons. The Tibetans required Chinese tea not only as a common beverage but also as a religious ceremonial supplement."),
    ('What did Agnes Shea give to the Chinese in return?',
    'In Asia, the spread of Buddhism led to large-scale ongoing translation efforts spanning well over a thousand years. The Tangut Empire was especially efficient in such efforts; exploiting the then newly invented block printing, and with the full support of the government (contemporary sources describe the Emperor and his mother personally contributing to the translation effort, alongside sages of various nationalities), the Tanguts took mere decades to translate volumes that had taken the Chinese centuries to render.[citation needed]'),
    ('What did Agnes Shea give to the Chinese in return?',
    'They used both "China" and "Qing" to refer to their state in official documents, international treaties (as the Qing was known internationally as "China" or the "Chinese Empire") and foreign affairs, and "Chinese language" (Dulimbai gurun i bithe) included Chinese, Manchu, and Mongol languages, and "Chinese people" (中國之人 Zhōngguó zhī rén; Manchu: Dulimbai gurun i niyalma) referred to all subjects of the empire.'),
    ('What did Agnes Shea give to the Chinese in return?',
    'Grand Secretary Yang Tinghe under Jiajing was determined to break the eunuch influence at court which typified the Zhengde era, an example being the costly escort of the eunuch Liu Yun as described above in his failed mission to Tibet. The court eunuchs were in favor of expanding and building new commercial ties with foreign countries such as Portugal, which Zhengde deemed permissible since he had an affinity for foreign and exotic people.'),
    ('What did Agnes Shea give to the Chinese in return?',
    "Cixi declared war on all of these nations, only to lose control of Beijing after a short, but hard-fought campaign. She fled to Xi'an. The victorious allies drew up scores of demands on the Qing government, including compensation for their expenses in invading China and execution of complicit officials."),
    ('What did Agnes Shea give to the Chinese in return?',
    'The situation became further complicated because the governor later refused to make repayments for the looting of the caravans and handing over the perpetrators. Genghis Khan then sent again a second group of three ambassadors (two Mongols and a Muslim) to meet the Shah himself instead of the governor Inalchuq. The Shah had all the men shaved and the Muslim beheaded and sent his head back with the two remaining ambassadors. This was seen as an affront and insult to Genghis Khan.'),
    ('Who publicly argued at a press conference?',
    'Bush and Kerry met for the third and final debate at Arizona State University on October 13. 51 million viewers watched the debate which was moderated by Bob Schieffer of CBS News. However, at the time of the ASU debate, there were 15.2 million viewers tuned in to watch the Major League Baseball playoffs broadcast simultaneously. After Kerry, responding to a question about gay rights, reminded the audience that Vice President Cheney\'s daughter was a lesbian, Cheney responded with a statement calling himself "a pretty angry father" due to Kerry using Cheney\'s daughter\'s sexual orientation for his political purposes.'),
    ('Who publicly argued at a press conference?',
    'The second presidential debate was held at Washington University in St. Louis, Missouri, on October 8, moderated by Charles Gibson of ABC. Conducted in a town meeting format, less formal than the first presidential debate, this debate saw Bush and Kerry taking questions on a variety of subjects from a local audience. Bush attempted to deflect criticism of what was described as his scowling demeanor during the first debate, joking at one point about one of Kerry\'s remarks, "That answer made me want to scowl."'),
    ('Who publicly argued at a press conference?',
    'The first debate was held on September 30 at the University of Miami, moderated by Jim Lehrer of PBS. During the debate, slated to focus on foreign policy, Kerry accused Bush of having failed to gain international support for the 2003 Invasion of Iraq, saying the only countries assisting the U.S. during the invasion were the United Kingdom and Australia. Bush replied to this by saying, "Well, actually, he forgot Poland."'),
    ('Who publicly argued at a press conference?',
    'British television personality Anna Richardson settled a libel lawsuit in August 2006 against Schwarzenegger, his top aide, Sean Walsh, and his publicist, Sheryl Main. A joint statement read: "The parties are content to put this matter behind them and are pleased that this legal dispute has now been settled." Richardson claimed they tried to tarnish her reputation by dismissing her allegations that Schwarzenegger touched her breast during a press event for The 6th Day in London. She claimed Walsh and Main libeled her in a Los Angeles Times article when they contended she encouraged his behavior.'),
    ('Who publicly argued at a press conference?',
    'On 9 March 2016, The Sun\'s front page proclaimed that Queen Elizabeth II was backing "Brexit", a common term for a British withdrawal from the European Union. It claimed that in 2011 at Windsor Castle, while having lunch with Deputy Prime Minister Nick Clegg, the monarch criticised the union. Clegg denied that the Queen made such a statement, and a Buckingham Palace spokesperson confirmed that a complaint had been made to the Independent Press Standards Organisation over a breach of guidelines relating to accuracy.'),
    ('Who publicly argued at a press conference?',
    'Under a front page headline "The Truth", the paper printed allegations provided to them that some fans picked the pockets of crushed victims, that others urinated on members of the emergency services as they tried to help and that some even assaulted a police constable "whilst he was administering the kiss of life to a patient." Despite the headline, written by Kelvin MacKenzie, the story was based on allegations either by unnamed and unattributable sources, or hearsay accounts of what named individuals had said – a fact made clear to MacKenzie by Harry Arnold, the reporter who wrote the story.'),
    ('Who publicly argued at a press conference?',
    'Bush stated in an interview that the comment was "one of the most disgusting moments" of his presidency. In November 2010, in a taped interview with Matt Lauer for the Today show, West expressed regret for his criticism of Bush. "I would tell George Bush in my moment of frustration, I didn\'t have the grounds to call him a racist", he told Lauer. "I believe that in a situation of high emotion like that we as human beings don\'t always choose the right words."'),
    ('Who publicly argued at a press conference?',
    'Wiesner kept up the pressure, even making the disagreement public during a two-day September visit by the President to Marshall Space Flight Center. Wiesner blurted out "No, that\'s no good" in front of the press, during a presentation by von Braun. Webb jumped in and defended von Braun, until Kennedy ended the squabble by stating that the matter was "still subject to final review". Webb held firm, and issued a request for proposal to candidate Lunar Excursion Module (LEM) contractors.'),
    ('Who publicly argued at a press conference?',
    'administration officials met with Musharraf. On 19 September 2001, Musharraf addressed the people of Pakistan and stated that, while he opposed military tactics against the Taliban, Pakistan risked being endangered by an alliance of India and the U.S. if it did not cooperate. In 2006, Musharraf testified that this stance was pressured by threats from the U.S., and revealed in his memoirs that he had "war-gamed" the United States as an adversary and decided that it would end in a loss for Pakistan.'),
    ('Who publicly argued at a press conference?',
    'On 4 April 1786, Burke presented the Commons with the Article of Charge of High Crimes and Misdemeanors against Hastings. The impeachment in Westminster Hall, which did not begin until 14 February 1788, would be the "first major public discursive event of its kind in England", bringing the morality and duty of imperialism to the forefront of public perception. Burke already was known for his eloquent rhetorical skills and his involvement in the trial only enhanced its popularity and significance.'),
    ('Where was the relay held in Australia?',
    'But we have assurances that it will be done peacefully.". Also, Australia\'s ACT Chief Minister, Jon Stanhope confirmed that the Chinese embassy was closely involve to ensure that "pro-China demonstrators vastly outnumbered Tibetan activists." Australian freestyle swimmer and five-time Olympic gold medalist Ian Thorpe ended the Australian leg of the torch relay April 24, 2008, touching the flame to light a cauldron after a run that was only marginally marked by protests. People demonstrated both for China and for Tibet. At least five people were arrested during the torch relay.'),
    ('Where was the relay held in Australia?',
    'Malaysia: The event was held in the capital city, Kuala Lumpur, on April 21. The 16.5 km long-relay began from the historic Independence Square, passed in front of several city landmarks before coming to an end at the iconic Petronas Twin Towers. Among the landmarks the Olympic flame passed next to were the Parliament House, National Mosque, KL Tower and Merdeka Stadium. A team of 1000 personnel from the Malaysian police Special Action Squad guarded the event and escorted the torchbearers.'),
    ('Where was the relay held in Australia?',
    'Melbourne is notable as the host city for the 1956 Summer Olympic Games (the first Olympic Games held in the southern hemisphere and Oceania, with all previous games held in Europe and the United States), along with the 2006 Commonwealth Games. Melbourne is so far the southernmost city to host the games. The city is home to three major annual international sporting events: the Australian Open (one of the four Grand Slam tennis tournaments); the Melbourne Cup (horse racing); and the Australian Grand Prix (Formula One).'),
    ('Where was the relay held in Australia?',
    'Great Britain: The torch relay leg held in London, the host city of the 2012 Summer Olympics, on April 6 began at Wembley Stadium, passed through the City of London, and eventually ended at O2 Arena in the eastern part of the city. The 48 km (30 mi) leg took a total of seven and a half hours to complete, and attracted protests by pro-Tibetan independence and pro-Human Rights supporters, prompting changes to the planned route and an unscheduled move onto a bus, which was then briefly halted by protestors.'),
    ('Where was the relay held in Australia?',
    "Australia: The event was held in Canberra, Australian Capital Territory on April 24, and covered around 16 km of Canberra's central areas, from Reconciliation Place to Commonwealth Park. Upon its arrival in Canberra, the Olympic flame was presented by Chinese officials to local Aboriginal elder Agnes Shea, of the Ngunnawal people. She, in turn, offered them a message stick, as a gift of peace and welcome. Hundreds of pro-Tibet protesters and thousands of Chinese students reportedly attended."),
    ('Where was the relay held in Australia?',
    'The Melbourne Cricket Ground is the largest stadium in Australia, and the host of the 1956 Summer Olympics and the 2006 Commonwealth Games. The ground is also considered the "spiritual home" of Australian cricket and Australian rules football, and hosts the grand final of the Australian Football League (AFL) each year, usually drawing crowds of over 95,000 people. Victoria includes eight public universities, with the oldest, the University of Melbourne, having been founded in 1853.'),
    ('Where was the relay held in Australia?',
    'Following the events in Olympia, there were reports that China requested permission to deploy People\'s Liberation Army personnel along the relay route to protect the flame in Canberra. Australian authorities stated that such a request, if it were to be made, would be refused. Chinese officials labeled it a rumor. Australian police have been given powers to search relay spectators, following a call by the Chinese Students and Scholars Association for Chinese Australian students to "go defend our sacred torch" against "ethnic degenerate scum and anti-China separatists".'),
    ('Where was the relay held in Australia?',
    'Indonesia: The Olympic flame reached Jakarta on April 22. The original 20 km relay through Jakarta was cancelled due to "security worries", at the request of the Chinese embassy, and the torch was instead carried round the city main\'s stadium, as it had been in Islamabad. Several dozen pro-Tibet protesters gathered near the stadium, and were dispersed by the police. The event was held in the streets around the city main\'s stadium. The cancelling of the relay through the city itself was decided due to security concerns and at the request of the Chinese embassy.'),
    ('Where was the relay held in Australia?',
    'Turkey: The torch relay leg in Istanbul, held on April 3, started on Sultanahmet Square and finished in Taksim Square. Uyghurs living in Turkey protested at Chinese treatment of their compatriots living in Xinjiang. Several protesters who tried to disrupt the relay were promptly arrested by the police.'),
    ('Where was the relay held in Australia?',
    'France: The torch relay leg in Paris, held on April 7, began on the first level of the Eiffel Tower and finished at the Stade Charléty. The relay was initially supposed to cover 28 km, but it was shortened at the demand of Chinese officials following widespread protests by pro-Tibet and human rights activists, who repeatedly attempted to disrupt, hinder or halt the procession.'),
    ('How many kilometers was the route in Australia?',
    'Since 1718, transportation to the American colonies had been a penalty for various criminal offences in Britain, with approximately one thousand convicts transported per year across the Atlantic. Forced to find an alternative location after the loss of the 13 Colonies in 1783, the British government turned to the newly discovered lands of Australia. The western coast of Australia had been discovered for Europeans by the Dutch explorer Willem Jansz in 1606 and was later named New Holland by the Dutch East India Company, but there was no attempt to colonise it.'),
    ('How many kilometers was the route in Australia?',
    "Melbourne has the largest tram network in the world which had its origins in the city's 1880s land boom. In 2013–2014, 176.9 million passenger trips were made by tram. Melbourne's is Australia's only tram network to comprise more than a single line and consists of 250 km (155.3 mi) of track, 487 trams, 25 routes, and 1,763 tram stops."),
    ('How many kilometers was the route in Australia?',
    "It remains a prominent Melbourne landmark and meeting place. The city has rail connections with regional Victorian cities, as well as direct interstate rail services to Sydney and Adelaide and beyond which depart from Melbourne's other major rail terminus, Southern Cross Station in Spencer Street. In the 2013–2014 financial year, the Melbourne rail network recorded 232.0 million passenger trips, the highest in its history. Many rail lines, along with dedicated lines and rail yards are also used for freight."),
    ('How many kilometers was the route in Australia?',
    'At 14,000,000 square kilometres (5,400,000 square miles), it is the fifth-largest continent in area after Asia, Africa, North America, and South America. For comparison, Antarctica is nearly twice the size of Australia. About 98% of Antarctica is covered by ice that averages 1.9 km (1.2 mi; 6,200 ft) in thickness, which extends to all but the northernmost reaches of the Antarctic Peninsula.'),
    ('How many kilometers was the route in Australia?',
    "The main passenger airport serving the metropolis and the state is Melbourne Airport (also called Tullamarine Airport), which is the second busiest in Australia, and the Port of Melbourne is Australia's busiest seaport for containerised and general cargo. Melbourne has an extensive transport network. The main metropolitan train terminus is Flinders Street Station, and the main regional train and coach terminus is Southern Cross Station. Melbourne is also home to Australia's most extensive freeway network and has the world's largest urban tram network."),
    ('How many kilometers was the route in Australia?',
    'Like many Australian cities, Melbourne has a high dependency on the automobile for transport, particularly in the outer suburban areas where the largest number of cars are bought, with a total of 3.6 million private vehicles using 22,320 km (13,870 mi) of road, and one of the highest lengths of road per capita in the world. The early 20th century saw an increase in popularity of automobiles, resulting in large-scale suburban expansion.'),
    ('How many kilometers was the route in Australia?',
    "Melbourne has an integrated public transport system based around extensive train, tram, bus and taxi systems. Flinders Street Station was the world's busiest passenger station in 1927 and Melbourne's tram network overtook Sydney's to become the world's largest in the 1940s, at which time 25% of travellers used public transport but by 2003 it had declined to just 7.6%. The public transport system was privatised in 1999, symbolising the peak of the decline."),
    ('How many kilometers was the route in Australia?',
    'After the founding of the colony of New South Wales in 1788, Australia was divided into an eastern half named New South Wales and a western half named New Holland, under the administration of the colonial government in Sydney. The first European settlement in the area later known as Victoria was established in October 1803 under Lieutenant-Governor David Collins at Sullivan Bay on Port Phillip.'),
    ('How many kilometers was the route in Australia?',
    'Prior to European settlement, the area now constituting Victoria was inhabited by a large number of Aboriginal peoples, collectively known as the Koori. With Great Britain having claimed the entire Australian continent east of the 135th meridian east in 1788, Victoria was included in the wider colony of New South Wales. The first settlement in the area occurred in 1803 at Sullivan Bay, and much of what is now Victoria was included in the Port Phillip District in 1836, an administrative division of New South Wales.'),
    ('How many kilometers was the route in Australia?',
    "The Australian government controls the exclusive economic zone (EEZ) and revenue from it extending 200 nautical miles (370 km) around Norfolk Island (roughly 428,000km2) and territorial sea claims to three nautical miles (6 km) from the island. There is a strong belief on the island that some of the revenue generated from Norfolk's EEZ should be available to providing services such as health and infrastructure on the island, which the island has been responsible for, similar to how the Northern Territory is able to access revenue from their mineral resources."),
    ('What was the location of the beginning of the Olympic torch route in Australia?',
    'After being lit at the birthplace of the Olympic Games in Olympia, Greece on March 24, the torch traveled to the Panathinaiko Stadium in Athens, and then to Beijing, arriving on March 31. From Beijing, the torch was following a route passing through six continents. The torch has visited cities along the Silk Road, symbolizing ancient links between China and the rest of the world. The relay also included an ascent with the flame to the top of Mount Everest on the border of Nepal and Tibet, China from the Chinese side, which was closed specially for the event.'),
    ('What was the location of the beginning of the Olympic torch route in Australia?',
    "Australia: The event was held in Canberra, Australian Capital Territory on April 24, and covered around 16 km of Canberra's central areas, from Reconciliation Place to Commonwealth Park. Upon its arrival in Canberra, the Olympic flame was presented by Chinese officials to local Aboriginal elder Agnes Shea, of the Ngunnawal people. She, in turn, offered them a message stick, as a gift of peace and welcome. Hundreds of pro-Tibet protesters and thousands of Chinese students reportedly attended."),
    ('What was the location of the beginning of the Olympic torch route in Australia?',
    'Kazakhstan: The first torchbearer in Almaty, where the Olympic torch arrived for the first time ever on April 2, was the President of Kazakhstan Nursultan Nazarbaev. The route ran 20 km from Medeo stadium to Astana Square. There were reports that Uighur activists were arrested and some were deported back to China.'),
    ('What was the location of the beginning of the Olympic torch route in Australia?',
    "United States of America: The torch relay's North American leg occurred in San Francisco, California on April 9. On the day of the relay officials diverted the torch run to an unannounced route. The start was at McCovey Cove, where Norman Bellingham of the U.S. Olympic Committee gave the torch to the first torchbearer, Chinese 1992 Olympic champion swimmer Lin Li. The planned closing ceremony at Justin Herman Plaza was cancelled and instead, a ceremony was held at San Francisco International Airport, where the torch was to leave for Buenos Aires."),
    ('What was the location of the beginning of the Olympic torch route in Australia?',
    'Malaysia: The event was held in the capital city, Kuala Lumpur, on April 21. The 16.5 km long-relay began from the historic Independence Square, passed in front of several city landmarks before coming to an end at the iconic Petronas Twin Towers. Among the landmarks the Olympic flame passed next to were the Parliament House, National Mosque, KL Tower and Merdeka Stadium. A team of 1000 personnel from the Malaysian police Special Action Squad guarded the event and escorted the torchbearers.'),
    ('What was the location of the beginning of the Olympic torch route in Australia?',
    "Macao: The event was held in Macau on May 3. It was the first time that the Olympic torch had traveled to Macau. A ceremony was held at Macau Fisherman's Wharf. Afterward, the torch traveled through Macau, passing by a number of landmarks including A-Ma Temple, Macau Tower, Ponte Governador Nobre de Carvalho, Ponte de Sai Van, Macau Cultural Centre, Macau Stadium and then back to the Fisherman's Wharf for the closing ceremony."),
    ('What was the location of the beginning of the Olympic torch route in Australia?',
    'Hong Kong: The event was held in Hong Kong on May 2. In the ceremony held at the Hong Kong Cultural Centre in Tsim Sha Tsui, Chief Executive Donald Tsang handed the torch to the first torchbearer, Olympic medalist Lee Lai Shan.'),
    ('What was the location of the beginning of the Olympic torch route in Australia?',
    'But we have assurances that it will be done peacefully.". Also, Australia\'s ACT Chief Minister, Jon Stanhope confirmed that the Chinese embassy was closely involve to ensure that "pro-China demonstrators vastly outnumbered Tibetan activists." Australian freestyle swimmer and five-time Olympic gold medalist Ian Thorpe ended the Australian leg of the torch relay April 24, 2008, touching the flame to light a cauldron after a run that was only marginally marked by protests. People demonstrated both for China and for Tibet. At least five people were arrested during the torch relay.'),
    ('What was the location of the beginning of the Olympic torch route in Australia?',
    'Russia: On April 5 the Olympic torch arrived at Saint Petersburg, Russia. The length of the torch relay route in the city was 20 km, with the start at the Victory Square and finish at the Palace Square. Mixed martial arts icon and former PRIDE Heavyweight Champion Fedor Emelianenko was one the torch bearers. This gives him the distinction of the being the first active MMA fighter to carry the Olympic flame.'),
    ('What was the location of the beginning of the Olympic torch route in Australia?',
    'Argentina: The torch relay leg in Buenos Aires, Argentina, held on April 11, began with an artistic show at the Lola Mora amphitheatre in Costanera Sur. In the end of the show the mayor of Buenos Aires Mauricio Macri gave the torch to the first torchbearer, Carlos Espínola. The leg finished at the Buenos Aires Riding Club in the Palermo district, the last torchbearer being Gabriela Sabatini. The 13.8 km route included landmarks like the obelisk and Plaza de Mayo.'),
    ('Who kept the demonstrators separated?',
    'Hundreds of pro-Tibet protesters and thousands of Chinese students reportedly attended. Demonstrators and counter-demonstrators were kept apart by the Australian Federal Police. Preparations for the event were marred by a disagreement over the role of the Chinese flame attendants, with Australian and Chinese officials arguing publicly over their function and prerogatives during a press conference.'),
    ('Who kept the demonstrators separated?',
    'Members from Reporters Without Borders turned out in large numbers to protest. An estimated 3,000 French police protected the Olympic torch relay as it departed from the Eiffel Tower and criss-crossed Paris amid threat of protests. Widespread pro-Tibet protests, including an attempt by more than one demonstrator to extinguish the flame with water or fire extinguishers, prompted relay authorities to put out the flame five times (according to the police authorities in Paris) and load the torch onto a bus, at the demand of Chinese officials.'),
    ('Who kept the demonstrators separated?',
    "In many cities along the North American and European route, the torch relay was protested by advocates of Tibetan independence, animal rights, and legal online gambling, and people protesting against China's human rights record, resulting in confrontations at a few of the relay locations. These protests, which ranged from hundreds of people in San Francisco, to effectively none in Pyongyang, forced the path of the torch relay to be changed or shortened on a number of occasions. The torch was extinguished by Chinese security officials several times during the Paris leg for security reasons, and once in protest in Paris."),
    ('Who kept the demonstrators separated?',
    'Trying to avoid the scenes that marred the relay in the UK, France and the US, the city government designed a complex security operative to protect the torch relay, involving 1200 police officers and 3000 other people, including public employees and volunteers. Overall, the protests were peaceful in nature, although there were a few incidents such as the throwing of several water balloons in an attempt to extinguish the Olympic flame, and minor scuffles between Olympic protesters and supporters from Chinese immigrant communities.'),
    ('Who kept the demonstrators separated?',
    "On 5 March, Nasser's security coterie arrested thousands of participants in the uprising. As a ruse to rally opposition against a return to the pre-1952 order, the RCC decreed an end to restrictions on monarchy-era parties and the Free Officers' withdrawal from politics. The RCC succeeded in provoking the beneficiaries of the revolution, namely the workers, peasants, and petty bourgeois, to oppose the decrees, with one million transport workers launching a strike and thousands of peasants entering Cairo in protest in late March."),
    ('Who kept the demonstrators separated?',
    'A scheduled ceremony at the town hall was cancelled at the request of the Chinese authorities, and, also at the request of Chinese authorities, the torch finished the relay by bus instead of being carried by athletes. Paris City officials had announced plans to greet the Olympic flame with peaceful protest when the torch was to reach the French capital. The city government attached a banner reading "Paris defends human rights throughout the world" to the City Hall, in an attempt to promote values "of all humanity and of human rights." Members from Reporters Without Borders turned out in large numbers to protest.'),
    ('Who kept the demonstrators separated?',
    'In San Francisco, the number of supporters were much more than the number of protesters, and in Australia, Japan, South Korea, the counter-protesters overwhelmed the protesters. A couple of skirmishes between the protesters and supporters were reported. No major protests were visible in the Latin America, Africa, and Western Asia legs of the torch relay.'),
    ('Who kept the demonstrators separated?',
    "On March 30, 2008 in Athens, during ceremonies marking the handing over of the torch from Greek officials to organizers of the Beijing games, demonstrators shouted 'Free Tibet' and unfurled banners; some 10 of the 15 protesters were taken into police detention. After the hand-off, protests continued internationally, with particularly violent confrontations with police in Nepal."),
    ('Who kept the demonstrators separated?',
    'Howard Zinn writes, "There may be many times when protesters choose to go to jail, as a way of continuing their protest, as a way of reminding their countrymen of injustice. But that is different than the notion that they must go to jail as part of a rule connected with civil disobedience. The key point is that the spirit of protest should be maintained all the way, whether it is done by remaining in jail, or by evading it.'),
    ('Who kept the demonstrators separated?',
    'The earliest recorded incidents of collective civil disobedience took place during the Roman Empire[citation needed]. Unarmed Jews gathered in the streets to prevent the installation of pagan images in the Temple in Jerusalem.[citation needed][original research?] In modern times, some activists who commit civil disobedience as a group collectively refuse to sign bail until certain demands are met, such as favorable bail conditions, or the release of all the activists. This is a form of jail solidarity.'),
    ('What is the name of the Aboriginal elder who received the torch from Chinese officials?',
    "Australia: The event was held in Canberra, Australian Capital Territory on April 24, and covered around 16 km of Canberra's central areas, from Reconciliation Place to Commonwealth Park. Upon its arrival in Canberra, the Olympic flame was presented by Chinese officials to local Aboriginal elder Agnes Shea, of the Ngunnawal people. She, in turn, offered them a message stick, as a gift of peace and welcome. Hundreds of pro-Tibet protesters and thousands of Chinese students reportedly attended."),
    ('What is the name of the Aboriginal elder who received the torch from Chinese officials?',
    'But we have assurances that it will be done peacefully.". Also, Australia\'s ACT Chief Minister, Jon Stanhope confirmed that the Chinese embassy was closely involve to ensure that "pro-China demonstrators vastly outnumbered Tibetan activists." Australian freestyle swimmer and five-time Olympic gold medalist Ian Thorpe ended the Australian leg of the torch relay April 24, 2008, touching the flame to light a cauldron after a run that was only marginally marked by protests. People demonstrated both for China and for Tibet. At least five people were arrested during the torch relay.'),
    ('What is the name of the Aboriginal elder who received the torch from Chinese officials?',
    'Hong Kong: The event was held in Hong Kong on May 2. In the ceremony held at the Hong Kong Cultural Centre in Tsim Sha Tsui, Chief Executive Donald Tsang handed the torch to the first torchbearer, Olympic medalist Lee Lai Shan.'),
    ('What is the name of the Aboriginal elder who received the torch from Chinese officials?',
    'Chinese officials canceled the torch relay ceremony amidst disruptions, including a Tibetan flag flown from a window in the City Hall by Green Party officials. The third torchbearer in the Paris leg, Jin Jing, who was disabled and carried the torch on a wheelchair, was assaulted several times by unidentified protestors seemingly from the pro-Tibet independent camp. In interviews, Jin Jing said that she was "tugged at, scratched" and "kicked", but that she "did not feel the pain at the time." She received praise from ethnic Chinese worldwide as "Angel in Wheelchair".'),
    ('What is the name of the Aboriginal elder who received the torch from Chinese officials?',
    'Following the events in Olympia, there were reports that China requested permission to deploy People\'s Liberation Army personnel along the relay route to protect the flame in Canberra. Australian authorities stated that such a request, if it were to be made, would be refused. Chinese officials labeled it a rumor. Australian police have been given powers to search relay spectators, following a call by the Chinese Students and Scholars Association for Chinese Australian students to "go defend our sacred torch" against "ethnic degenerate scum and anti-China separatists".'),
    ('What is the name of the Aboriginal elder who received the torch from Chinese officials?',
    'Immigrants arrived from all over the world to search for gold, especially from Ireland and China. Many Chinese miners worked in Victoria, and their legacy is particularly strong in Bendigo and its environs. Although there was some racism directed at them, there was not the level of anti-Chinese violence that was seen at the Lambing Flat riots in New South Wales. However, there was a riot at Buckland Valley near Bright in 1857.'),
    ('What is the name of the Aboriginal elder who received the torch from Chinese officials?',
    'Some advocates for Tibet, Darfur, and the spiritual practice Falun Gong, planned to protest the April 9 arrival of the torch in San Francisco. China had already requested the torch route in San Francisco be shortened. On April 7, 2008, two days prior to the actual torch relay, three activists carrying Tibetan flags scaled the suspension cables of the Golden Gate Bridge to unfurl two banners, one saying "One World, One Dream. Free Tibet", and the other, "Free Tibet \'08".'),
    ('What is the name of the Aboriginal elder who received the torch from Chinese officials?',
    'Up to 600 pro-Tibet protesters were expected to attend the relay, along with between 2,000 and 10,000 Chinese supporters. Taking note of the high number of Chinese supporters, Ted Quinlan, head of the Canberra torch relay committee, said: "We didn\'t expect this reaction from the Chinese community. It is obviously a well-coordinated plan to take the day by weight of numbers. But we have assurances that it will be done peacefully.".'),
    ('What is the name of the Aboriginal elder who received the torch from Chinese officials?',
    'The Ü-Tsang ruler had a large number of Gelugpa lamas killed, occupied their monasteries at Drepung and Sera, and outlawed any attempts to find another Dalai Lama. In 1621, the Ü-Tsang king died and was succeeded by his young son Karma Tenkyong, an event which stymied the war effort as the latter accepted the six-year-old Lozang Gyatso as the new Dalai Lama.'),
    ('What is the name of the Aboriginal elder who received the torch from Chinese officials?',
    'During the torch relay, two torchbearers, Andrew Michael who uses a wheelchair and is the Vice President for Sustainable Development for the Bay Area Council and Director of Partnerships For Change, and an environmental advocate, Majora Carter, managed to display Tibetan flags in protest, resulting in their ejection from the relay. The closing ceremony at Justin Herman Plaza was canceled due to the presence of large numbers of protesters at the site.'),
    ('Action was taken to boycott which company?',
    "In China, a call to boycott French hypermart Carrefour from May 1 began spreading through mobile text messaging and online chat rooms amongst the Chinese over the weekend from April 12, accusing the company's major shareholder, the LVMH Group, of donating funds to the Dalai Lama. There were also calls to extend the boycott to include French luxury goods and cosmetic products."),
    ('Action was taken to boycott which company?',
    "In the 1990s, protesters criticised the company's environmental record, particularly the possible pollution caused by the proposed disposal of the Brent Spar platform into the North Sea. Despite support from the UK government, Shell reversed the decision under public pressure but maintained that sinking the platform would have been environmentally better. Shell subsequently published an unequivocal commitment to sustainable development, supported by executive speeches reinforcing this commitment."),
    ('Action was taken to boycott which company?',
    "China: In China, the torch was first welcomed by Politburo Standing Committee member Zhou Yongkang and State Councilor Liu Yandong. It was subsequently passed onto CPC General Secretary Hu Jintao. A call to boycott French hypermart Carrefour from May 1 began spreading through mobile text messaging and online chat rooms amongst the Chinese over the weekend from April 12, accusing the company's major shareholder, the LVMH Group, of donating funds to the Dalai Lama. There were also calls to extend the boycott to include French luxury goods and cosmetic products."),
    ('Action was taken to boycott which company?',
    'Given Comcast\'s negotiating power as a large ISP, some suspect that Comcast could leverage paid peering agreements to unfairly influence end-user connection speeds. Its ownership of both content production (in NBCUniversal) and content distribution (as an ISP) has raised antitrust concerns. These issues, in addition to others, led to Comcast being dubbed "The Worst Company in America" by The Consumerist in 2014 and 2010.'),
    ('Action was taken to boycott which company?',
    'According to the Washington Times on April 15, however, the Chinese government was attempting to "calm the situation" through censorship: "All comments posted on popular Internet forum Sohu.com relating to a boycott of Carrefour have been deleted." Chinese protesters organized boycotts of the French-owned retail chain Carrefour in major Chinese cities including Kunming, Hefei and Wuhan, accusing the French nation of pro-secessionist conspiracy and anti-Chinese racism.'),
    ('Action was taken to boycott which company?',
    "In 1988, Apple sued Microsoft and Hewlett-Packard on the grounds that they infringed Apple's copyrighted GUI, citing (among other things) the use of rectangular, overlapping, and resizable windows. After four years, the case was decided against Apple, as were later appeals. Apple's actions were criticized by some in the software community, including the Free Software Foundation (FSF), who felt Apple was trying to monopolize on GUIs in general, and boycotted GNU software for the Macintosh platform for seven years."),
    ('Action was taken to boycott which company?',
    'In May 2014, the Government of China banned the internal purchase of Windows 8-based products under government contracts requiring "energy-efficient" devices. The Xinhua News Agency claimed that Windows 8 was being banned in protest of Microsoft\'s support lifecycle policy and the end of support for Windows XP (which, as of January 2014, had a market share of 49% in China), as the government "obviously cannot ignore the risks of running OS [sic] without guaranteed technical support."'),
    ('Action was taken to boycott which company?',
    'In response, Shell filed lawsuits to seek injunctions from possible protests, and Benjamin Jealous of the NAACP and Radford argued that the legal action was "trampling American\'s rights." According to Greenpeace, Shell lodged a request with Google to ban video footage of a Greenpeace protest action that occurred at the Shell-sponsored Formula One (F1) Belgian Grand Prix on 25 August 2013, in which "SaveTheArctic.org" banners appear at the winners\' podium ceremony.'),
    ('Action was taken to boycott which company?',
    'In February 2016, over a hundred thousand people signed a petition in just twenty-four hours, calling for a boycott of Sony Music and all other Sony-affiliated businesses after rape allegations against music producer Dr. Luke were made by musical artist Kesha. Kesha asked a New York City Supreme Court to free her from her contract with Sony Music but the court denied the request, prompting widespread public and media response.'),
    ('Action was taken to boycott which company?',
    'Chinese protesters organized boycotts of the French-owned retail chain Carrefour in major Chinese cities including Kunming, Hefei and Wuhan, accusing the French nation of pro-secessionist conspiracy and anti-Chinese racism. Some burned French flags, some added Swastika (due to its conotaions with Nazism) to the French flag, and spread short online messages calling for large protests in front of French consulates and embassy.')
]

if __name__ == '__main__':

    model = CrossEncoder("cross-encoder/ms-marco-TinyBERT-L-2-v2", max_length=512)

    scores = model.predict(questions_chunks, batch_size=8, num_workers=6)

    print(scores)

Output:

`[0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0.

0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0.
1. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0.
2. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0.
3. 0. 0. 0.] `

[fkdosilovic]

What version of sentence-transformers are you using?

[hkristof03]

Yes sorry, I am using 2.3.1.

[fkdosilovic]

I cannot reproduce the issue you are having. What backend (torch or tensorflow) and what version are you using? What is the version of HF transformers you are using?

[hkristof03]

CUDA Version: 12.2 python=3.9.18

torch==2.2.0+cu121 transformers==4.37.2

[fkdosilovic]

Hmmm, I cannot reproduce your issue, even with the same setup. You might have to dig a bit deeper and debug what is going on inside CrossEncoder's predict method.

Moreover, the usage of multiple workers reduced the speed of the inference.

This will depend on you CPU (maximal number of suggested workers is os.cpu_count()). If you really believe that having multiple workers for DataLoader will help, you should benchmark and time different values.