Category: Tech

Using Artificial Intelligence (AI) chatbots to seek medical advice can expose patients to risks, a new study has found.

The research concluded that AI tools used for medical decision-making frequently produce inaccurate and inconsistent information, which can lead to incorrect diagnoses and inappropriate advice. The study was conducted by researchers from the Oxford Internet Institute and the Nuffield Department of Primary Care Health Sciences at the University of Oxford and was published in the journal Nature Medicine.

The findings raise concerns as large language model-based chatbots are increasingly used by people seeking guidance on symptoms and possible health conditions.

Dr Rebecca Payne, a general practitioner and co-author of the study, said that the research showed that AI systems are not ready to function as a substitute for medical professionals. “Despite all the hype, AI just isn’t ready to take on the role of the physician,” she said.

She cautioned that people who rely on AI-generated medical responses may be put at risk. “Patients need to be aware that asking a large language model about their symptoms can be dangerous, giving wrong diagnoses and failing to recognise when urgent help is needed,” Dr Payne added.

As part of the study, researchers asked nearly 1,300 participants to assess a series of health-related scenarios. Participants were required to identify possible conditions and decide what action should be taken.

Some participants used AI tools powered by large language models to receive suggested diagnoses and guidance on next steps, while others relied on traditional methods, including consulting a GP.

Researchers evaluated the responses and found that AI-generated advice often combined correct information with incorrect or misleading guidance. The study noted that many users struggled to determine which parts of the information provided by the AI could be trusted.

Although the study found that AI chatbots perform well in standardised medical knowledge tests, researchers said this did not translate into safe or reliable use in real-world health situations. The research warned that using AI systems to assess personal medical symptoms could place users at risk.

“These findings highlight the difficulty of building AI systems that can genuinely support people in sensitive, high-stakes areas like health,” Dr Payne said.

The study’s lead author, Andrew Bean of the Oxford Internet Institute said that the results demonstrated ongoing limitations in how AI systems interact with people.

A whistleblower complaint filed with the US Securities and Exchange Commission alleges that Google provided assistance in 2024 to an Israeli military contractor despite company policies that restricted the use of artificial intelligence for weapons and surveillance.

The complaint, submitted by a Google employee who no longer works at the company, was filed on a confidential basis and reviewed by US media outlets, including The Washington Post. It alleges that Google’s cloud services division responded to a customer support request linked to an Israel Defense Forces email address and an employee of CloudEx, an Israeli technology firm described in the complaint as working with Israel’s defense establishment.

According to the filing, the support request was made in July 2024 and sought help improving the performance of Google’s Gemini AI system in identifying drones, vehicles and people in aerial video footage. Internal documents attached to the complaint state that Google staff provided guidance and carried out internal testing related to the request.

At the time, Google’s publicly stated AI principles said the company would not apply artificial intelligence to weapons-related purposes or to surveillance uses that could violate accepted norms. The whistleblower argues that the assistance described in the documents conflicted with those principles.

The complaint further alleges that Google may have violated securities laws by including its AI principles in public disclosures while acting in ways that were not consistent with them. It states that this may have misled investors and regulators about how the company applied its policies.

The whistleblower filed the complaint anonymously, citing concern about retaliation. The filing states that Google’s internal AI review process was applied in most cases but was not enforced in the same way in matters involving Israel and the genocide in Gaza.

Israel has been accused of committing  multiple war crimes in Gaza including the indiscriminate killing of civilians. Palestinians in Gaza have alleged that the drones spied on civilian families and led to strikes that killed scores. 

Google denied the allegations. A company spokesperson said the interaction amounted to “a general support response comparable to what any customer would receive” and did not involve specialized technical assistance. The spokesperson also said the account involved showed “minimal spending on AI services,” adding that such usage would not allow meaningful deployment of the technology.

TikTok’s United States (US) app has said that recent disruptions, including problems sending messages containing the name “Epstein” and alleged suppression of content related to the Immigration and Customs Enforcement agency (ICE), were caused by a power outage at one of its data centers.

The company pushed back against claims that the issues were politically motivated, saying the disruptions were linked to technical problems rather than censorship.

Earlier, users reported account glitches after TikTok began operating under a US-led joint venture, following months of scrutiny over its Chinese ownership. 

Some users claimed messages with the word “Epstein” failed to send, prompting warnings that the messages might violate community guidelines.

A spokesperson for the TikTok joint venture told CNBC that the platform “does not prohibit sharing the name ‘Epstein’ in messages” and said the company was investigating why some users were experiencing problems.

The concern surrounding the messaging issue relates to Jeffrey Epstein, the late financier and convicted sex offender. Since December, the Department of Justice has been releasing portions of documents tied to its investigations into Epstein but has not released the entirety of the so-called “Epstein files”.

Complaints about TikTok’s handling of content have also focused on videos related to ICE. The agency has been involved in enforcement operations in Minneapolis that led to the fatal shootings of two US citizens. 

Some users posted videos flagged as “Ineligible for Recommendation”, leading to claims that anti-Trump and anti-ICE content was being restricted.

California Governor Gavin Newsom amplified these concerns, stating in a Tuesday post on X that his “office has received reports and independently confirmed instances of suppressed content critical of President Trump”.

His office said it was launching a review and asked the California Department of Justice to determine whether TikTok’s conduct violated California law. 

Hours before Newsom’s statement, TikTok posted on X that the company was “continuing to resolve” a major infrastructure issue.

“While the network has been recovered, the outage caused a cascading systems failure that we’ve been working to resolve with our data center partner,” the company said.

It added that users might notice bugs, slower load times and timeouts when posting content, and that some videos could temporarily show “0” views or likes.

When asked directly about censorship claims, including ICE-related content, a joint venture spokesperson said that videos of the Minneapolis incident were available on the platform and had been since Saturday.

TikTok has announced establishing a majority American-owned joint venture to operate its United States (US) business, a move aimed at avoiding bans linked to its Chinese ownership.

The TikTok Joint Venture will serve over 200 million users and 7.5 million businesses in the US while implementing safeguards for data protection and content moderation, the company said.

The new structure comes in response to a 2024 law under former president Joe Biden, which required Chinese-owned ByteDance to sell TikTok’s US operations or face a ban. The law was introduced amid concerns that China could access US user data or influence audiences through TikTok’s algorithm.

US President Donald Trump has welcomed the deal and credited himself for facilitating it, while also acknowledging Chinese President Xi Jinping for approving the arrangement. “I am so happy to have helped in saving TikTok!” Trump wrote on Truth Social. 

“It will now be owned by a group of Great American Patriots and Investors, the Biggest in the World, and will be an important Voice,” he said.

Under the new venture, ByteDance retains a 19.9 percent stake, keeping its ownership below the 20 percent limit set by law. Three other investors Silver Lake, Oracle and Abu Dhabi-based AI fund MGX each hold 15 percent stakes. 

The joint venture will have decision-making authority over trust and safety policies and content moderation for US users. TikTok’s global entities will manage international product integration and commercial activities, including e-commerce and advertising.

US user data will be stored in Oracle’s cloud environment and undergo third-party cybersecurity audits following federal standards, TikTok said.