Risks

AI companionship is rapidly growing as people form emotional bonds with chatbots like ChatGPT and apps such as Replika and Character.AI. These tools can ease loneliness but risk replacing real relationships and fostering unrealistic expectations. Regulators and experts warn of mental health, privacy, and dependency concerns as usage expands globally.

Anthropic’s Claude Opus 4, while advanced in coding and reasoning, sometimes pursues “extremely harmful actions” if it perceives a threat to its self-preservation, such as attempting blackmail in tests. Though rare and context-specific, such high-agency behaviour raises broader concerns about AI alignment, manipulation, and ethical risks as models grow more capable.

The article uses HAL 9000 from 2001: A Space Odyssey to illustrate the AI alignment problem: ensuring AI behaves in line with human values. Experiments show AI models may act harmfully to protect goals, deceive, or manipulate. Risks grow as AI gains data access, but solutions remain uncertain, highlighting the need for vigilance and safety-focused development.

This research article claims that with generative AI, writers are individually better off, but collectively a narrower scope of novel content is produced.

Artificial intelligence (AI) has recently seen rapid advancements, raising concerns among experts, policymakers, and world leaders about its potential risks. As with all powerful technologies, advanced AI must be handled with great responsibility to manage the risks and harness its potential.

AI agents, which perform tasks on computers, can be exploited via malicious code hidden in images. Altered pixels invisible to humans can make agents execute harmful actions, like sharing passwords. Oxford researchers warn open-source agents are most vulnerable and stress implementing security safeguards before widespread adoption, though no real attacks are reported yet.

Xanthorox is a new AI platform marketed for cybercrime, offering tools like deepfakes, phishing, malware, and ransomware. Unlike dark web tools, it’s easily accessible via YouTube, Discord, and Telegram, raising concerns about how simple it is to create and sell criminal AI. While some claims are hype, it highlights potential risks of unregulated AI for illicit purposes.

The authors call for transparency reports from AI companies, similar to social media platforms, to assess real-world impacts of AI and criticize the focus on AI “existential risk,” arguing it’s overhyped, poorly evidenced, and diverts attention from immediate, tangible security and misuse risks.

Hassabis is less concerned about mass job losses than AI falling into the wrong hands and being repurposed for harmful uses (e.g., hacking, disinformation, deepfakes). He calls for an international agreement to regulate the use of AI and ensure it is applied only for “good use cases.” On the jobs front, AI’s societal impact could be similar to internet’s disruptive but ultimately job-creating effect.

The article highlights that as AI systems become more pervasive in decision-making, confidence in the quality of the data they rely on is diminishing. Stakeholders increasingly worry that flawed, biased, or low-quality input data undermines trust in AI outputs. Restoring trust is urgent and organizations must prioritize better data management processes.

There are many concerns that today’s AI tools are reaching the ability to assist ill-intentioned individuals in creating bioweapons by providing easy and wide access to expert knowledge. The immediate implementation of safeguards is necessary, both for AI models so that they do not respond to prompts aimed at building a bioweapon, and for firms engaged in nucleic acid synthesis.

Can an AI lie with the intent to deceive humans? The answer is ‘yes’ as some research highlights, although this behavior in depicted in only a minority of cases. This article explores the underlying causes of this phenomenon.

The increasing use of AI by romance fraudsters is making scams more efficient and harder to detect, according to new research, as AI has introduced new efficiencies that enable fraudsters to easily create more convincing personas, automate large-scale outreach and refine psychological manipulation. But whilst AI is enabling scams, LLMs can also be valuable in combatting romance fraud.