Researchers from top US universities warn extending pre-training can be detrimental to performance
Too much pre-training can deliver worse performance due to something akin to the butterfly effect
The more they are pre-trained, the more they become sensitive to small changes that could disrupt the end result
Researchers from Carnegie Mellon, Stanford, Harvard, and Princeton are challenging one of AI development’s accepted core beliefs - that the more pre-training data the better the performance.
As reported by HPCwire, a new paper discuses the concept of “catastrophic overtraining,” whereby extended pre-training can harm a model’s performance after fine-tuning.
The researchers compared two versions of the OLMo-1B model, one trained on 2.3 trillion tokens and another on 3 trillion. Despite the larger training set, the more extensively trained model reportedly performed up to 3% worse on benchmarks like AlpacaEval and ARC.
Reaching the inflection point
This performance drop, the study claims, is linked to a phenomenon called “progressive sensitivity.”
As the token count increases, the model becomes more fragile. Even small tweaks, like adjustments during fine-tuning, or the introduction of noise, can reverse earlier gains.
The authors demonstrated this by injecting Gaussian noise into pre-trained models, noting that performance degraded more sharply the longer the model was trained.
The point where this additional training starts to degrade performance is called the “inflection point.”
Once reached, the benefits of training start to become outweighed by the risk of internal instability. The study found that this tipping point often occurs beyond 2.5 trillion tokens in smaller models, like OLMo-1B.
“Catastrophic overtraining may be inevitable... especially when the pre-training and fine-tuning tasks are misaligned,” the authors warn in their paper, which you can access through the arXiv pre-print server.
While the researchers are not suggesting an end to pre-training, they do feel that developers should consider just how much pre-training is enough. As the paper concludes, “Our findings call for a renewed focus on model scaling that considers the entire training pipeline.”
For AI developers chasing scale, the message seems clear: sometimes, less really is more.
Certain electronics gadgets are now exempt from certain tariffs
Phones, laptops, and processors all get a reprieve – which includes China
It's still not clear what will happen next with electronics pricing
There's a new twist in the US tariff trade war: Phones, computers, semiconductors and various other electronics are now exempt from punishing import tariffs, which have shot up to 145% for China and are typically now 10% as a baseline for other countries.
The news was reported by the Associated Press and others, and while it's still early to say what the full impact will be, it does ease the pressure on companies who mostly manufacture their gadgets outside of the US – including Apple, Dell, Nvidia, and Samsung.
US President Donald Trump introduced a raft of tariffs for goods imported to the US at the start of the month, and markets have been in chaos ever since. Earlier this week, a lot of those tariff hikes above the 10% baseline were paused – but not for China, which raised its own tariffs in turn.
This new exclusion policy does cover China, and mentions smartphones, laptops, hard drives, computer processors, memory chips, and machines used to make semiconductors. Most companies that deal in electronics will be breathing a sigh of relief, though some device categories apparently aren't covered, including video games consoles.
What happens next?
The move might not be enough to open up Switch 2 pre-orders in the US(Image credit: Future/Nintendo)
It's been difficult to predict the moves the US government has made in respect to tariffs in recent days, and that hasn't changed. This latest move should mean gadget pricing settles down for consumers, but it's impossible to be certain.
Bloomberg reports that the new exclusion policy could be a precursor to a whole separate tariff focused specifically on electronics. For now, the world is waiting to see how the 90-day pause on many of Trump's tariffs plays out.
As a US company which assembles almost all of its goods overseas, including in China, Apple has been in the spotlight during all of this tariff uncertainty: there have even been suggestions of Apple factories in the US and reports of iPhone panic-buying ahead of potential price rises.
Apple is one of the companies that it seems will benefit the most, though the move may not be enough to open up Nintendo Switch 2 pre-orders in the US, which have been delayed. Consoles like the Switch 2 aren't mentioned in the new exemptions, though the previous 90-day pause should mean more stock can be moved to the United States.
If the bond selloff continues, it could bring widespread economic pain in the form of higher borrowing costs on loans and credit, plus a slowdown in growth. Experts say for now it's 'wait and see.'
I love coffee, but I also love my sleep, so after about 2pm I always switch to decaf to avoid being kept awake at night. It works well, but opting for decaf generally means you miss out on some of the more unusual flavors around – like the double-fermented passionfruit beans I got from my local coffee roaster recently, or Nespresso's white chocolate and strawberry coffee pods, which are an unlikely but delicious combination of flavors.
My Speciality Coffee Association (SCA) instructor calls it 'cheating coffee', but when it tastes this good, who cares?
Thankfully, just as alcohol-free beer is now mainstream and varied, we're starting to see roasters and manufacturers start to get more creative with decaf – and Nespresso's new Sweet Vanilla Decaffeinato pods are so comforting, they might just become my new favorite bedtime drink.
The Sweet Vanilla Decaffeinato pods work in any Neespresso Vertuo machine(Image credit: Future)
Coffee beans can have notes of vanilla by themselves, depending on the variety and the roast, and when extracted correctly (a tricky process to get right), coffee does have a natural sweetness. However, it's quite subtle, and if you want something more dessert-like, a coffee with added flavor is the way forward.
Decaffeinated coffee is made by removing the caffeine from green (unroasted ) coffee beans by dissolving it in water. There are a few different ways to achieve this, and Nespresso uses two different methods. The first involves simply soaking the beans in hot water to dissolve the caffeine (known as the Swiss water process). This process leaves behind no residue that could alter the taste and character of the coffee
The second method (the carbon dioxide process) is more efficient. Again, it involves soaking the beans in water to make them porous, but this time the soaked beans are placed in a pressurized container and exposed to CO2, which dissolves the caffeine.
The vanilla flavor works well with barista oat milk(Image credit: Future)
After loading a pod into my Nespresso Vertuo Pop (one of the best Nespresso machines around if you need something compact) and hitting the brew button, I was left with a cup full of creamy decaf coffee with a generous layer of foam.
It's delicious by itself, and the added flavor doesn't overpower the taste of the beans, but I enjoy a milky bedtime drink and vanilla typically works well with dairy, so I was keen to see how it would hold up as a white coffee.
The Sweet Vanilla Decaffeinato pods are mug-sized, meaning they'll fill a 230ml mug, but I still had room for a little caramel-flavored barista oat milk, or regular dairy milk. The flavors both combine very nicely, though regular milk would work best if you don't have a particularly sweet tooth.
Hopefully we'll see even more decaf options soon – cheating or otherwise.
from Latest from TechRadar US in News,opinion https://ift.tt/umjYPL0
This week we've got River City Girls, Idle Champions of the Forgotten Realms and more. Check out how you can get these games and all their rewards at the Epic Games Store.
As someone who's covered Apple for almost a quarter of a century, this Apple Intelligence Siri chapter is by far its most confounding one. We're now just two months away from the next Apple developer's conference, and it will likely mark an ignominious anniversary for Apple's artificial intelligence efforts.
Apple's uncharacteristic failure to deliver on a product promise has shocked many fans. Ever since the company announced it would not deliver some of the promised Siri enhancements this year, I, for one, have been wondering how we got here.
Now, a new report paints a surprising picture of rival AI factions or rather two teams with divergent strategies that somehow led to flashy demos and some unfulfilled promises.
However, The Information's claim that Apple's AI lead John Giannandrea and Craig Federighi (Apple's software engineering head) were in some fashion acting at cross purposes is unexpected.
Giannandrea's slow and steady approach and Apple's insistence on using its own models as opposed to, say, OpenAI's GPT LLMs was in stark contrast to the obvious excitement Federighi's Intelligent Systems team had for experimenting with and possibly using third-party models.
Two visions, no clarity
(Image credit: Future / Lance Ulanoff)
Internal battles are not unheard of in companies of all sizes, but they usually don't play out on a public stage at Apple. As The Information tells it, the Apple Intelligence demos I saw last June at WWDC may have broken what I thought was a cardinal rule for Apple: never announce technology that you can't demo live, at least in some fashion.
When Apple stunned WWDC 2023 with the new Apple Vision Pro, it followed later that afternoon by inviting a handful of journalists (including me) for hands-on demos. The pricey mixed reality headset wasn't done, but it was still functional enough to blow our collective minds.
In the case of Apple's Intelligence, though, our post-event demos were limited (Siri was no smarter). It seems that not only were many of the best bits not finished, but Apple may have shown them off in the pre-tapped keynote without giving the main Apple AI team in charge of Siri a full heads-up.
Doing so obviously put Apple in a tough spot. The pre-tapped WWDC video showed us a responsive and aware Siri capable of looking into your iPhone, its apps, and your information to take action on your behalf. Apple then promised we'd see it sometime that year.
Even after Apple delivered the iPhone 16 with iOS 18 and the first blush of Apple Intelligence (a new look, writing tools, Image Playground, Genmojis), most assumed it was just a matter of time before Siri got its major intelligence glow-up.
Half baked
(Image credit: Future / Lance Ulanoff)
By possibly demonstrating technology that was not ready or even fully programmed to run on any Apple Intelligence-ready iPhone, Apple may have set itself or, at least the Siri team, up for failure.
Federighi's team, according to The Information, is much more open to integrating third-party LLMs. It's a good strategy not only because it will help Apple's AI efforts get back on track but also because it's the same strategy employed by most of its AI competitors, at least those who are not at heart AI companies.
Late in the game, Amazon relied on Perplexity's Claude AI models to build its own Nova system. Samsung has leaned heavily on Google Gemini to develop its own models. As we well know, Microsoft made a big bet on OpenAI's GPT models to power Bing AI and then Copilot to build its own Microsoft AI models.
Siri will rise
(Image credit: Future/Jacob Krol)
This, folks, is the way in. Sure, it's maybe less predictable and, possibly, rock-solid secure than Apple's initial strategy, but most of these companies are racing ahead while iPhone fans wait for Apple to get its act together.
Federighi (and Vision Pro lead Mike Walker) taking over is good news, but much of Apple's future success in the AI space will depend on Apple CEO Tim Cook's leadership.
This failure happened under his watch, possibly because he trusted the team to get the job done and has been adamant about sticking to Apple's secure and private approach. Now Cook has to find a new plan and, with Federighi, identify a clear path that gets you the smarter Siri you want in 2026, if not sooner.
I've seen Apple do amazing things in the past, and honestly, I'm confident it'll do it again here.
Moreover, ransomware is often talked about as an attack that businesses should expect to be hit by, with two-thirds of organizations experiencing an attack in the last two years.
But it’s not all doom and gloom, as new research from backup storage provider Object First shows that 81% of IT decision makers agree that immutable backup storage is the best ransomware protection.
TransUnion is a credit monitoring service that helps you stay on top of your financial health. With real-time alerts, credit score tracking, and identity theft protection, it ensures you never miss important changes. You'll benefit from a customizable online interface with clear insights into your credit profile. Businesses also benefit from TransUnion’s advanced risk assessment tools.
While having a 3-2-1 backup strategy is a start when it comes to ransomware protection, the Object First research also reveals that 96% of organizations who have suffered a ransomware attack in the last two years saw their data backups targeted as part of at least one attack, with one in ten seeing their backup storage targeted in every attack.
While there is a general consensus that immutable backup storage is probably the best way to protect against ransomware, only 59% of organizations actually deploy this form of storage, and only 58% have multiple copies of their data in separate locations. In addition to this, 61% of respondents believe that IT security hardening offers enough protection against ransomware attacks.
Object First recommends that businesses adopt a breach mentality, and couple this with Zero Trust principles and immutable backup storage. To do this, businesses should segment their networks to minimize the potential blast radius of an attack, alongside using authentication mechanisms for log-ins such as multi-factor authentication and identity access management practices. Moreover, each user should only have access to the files and applications they need for their specific workload.
Immutable backup storage helps protect against ransomware attacks by storing data in a way that it cannot be modified or deleted. Traditionally, these have been hard drives or other storage mediums that are used to back up data and then stored in an ‘air-gapped’ environment that has no connection to the outside world. However, with cloud services booming, immutable cloud storage is now also an option.
Google unveils combined security suite at Cloud Next 25
Google Unified Security brings together all its top safety tools
Threat Intelligence from Mandiant will help your business spot threats before they strike
Google Cloud has unveiled a new unified security platform designed to take the stress out of keeping your business safe from the latest threats.
The company has unveiled Google Unified Security, affectionately known as GUS, at its Google Cloud Next 25 event in Las Vegas, promising a major step forward in threat detection and mitigation.
Unsurprisingly, Google Unified Security will be outfitted with the company's latest AI tools and services to maximize the options available to customers.
Google Unified Security
The launch of Google Unified Security comes as businesses around the globe face what the company calls a "new era of security and privacy".
The company says that as a business grows, its infrastructure inevitably becomes more complex, expanding the possible attack surface, and increasing the need for a unified security platform, as separate and disconnected security tools can leave data fragmented, as well as opening firms up to attack.
Customers will be able to use Google Unified Security to access a single interface for all their data, offering improving visibility, detection and response capabilities.
The company promises the latest Google Threat Intelligence findings will be made available to users, and Gemini will of course also be present to offer advice and guidance.
Users will be able to access telemetry and other data from Chrome Enterprise to ensure they stay safe from possible threats, which can be monitored from Security Command Center.
Employees can also benefit from new phishing protections in Chrome Enterprise Premium, with employers able to add their own branding and corporate assets in order to boost identification.
Google's Security Operations platform is also getting a boost thanks to new data pipeline management capabilities, helping users manage scale and reduce costs, while the new Mandiant Threat Defense service gives in-depth detection and response capabilities to all the latest threats.
Ulefone Armor 30 Pro combines ruggedness with productivity
A triple-camera setup in a rugged smartphone isn't that common
64MP camera and a 4W speaker reflect a shift in what rugged smartphones aim to offer
The best rugged smartphones often focus on strength and battery life, but in recent years, they’ve started to include features such as portable projectors and even dual displays.
The Ulefone Armor 30 Pro is a dual-screen rugged device featuring a 6.95-inch FHD+ main display with a 120Hz refresh rate. On the rear, it sports a 3.4-inch secondary screen with a 960x412 resolution, protected by Gorilla Glass.
While the displays can’t be operated independently, both run on Android, and the rear screen allows users to take selfies and video calls using the main camera system.
Big battery and a MediaTek processor
The device is powered by the MediaTek Dimensity 7050 chipset, paired with 16GB of RAM and 512GB of internal storage. It supports 5G connectivity and AI-powered features.
As with many rugged devices, it houses a large 12,800mAh battery that promises extended usage. It supports 66W fast charging for quick top-ups and offers 10W reverse charging to power smaller devices like smartwatches or earbuds.
One major surprise feature is the Ulefone Armor 30 Pro's triple rear camera setup - which is pretty uncommon in this category. It includes a 50MP main sensor, a 50MP ultra-wide shooter, and a 64MP infrared night vision camera supported by four IR LED blasters.
The Armor 30 Pro is built for more than just survival in harsh environments. It features pogo pin connectors for accessories like endoscopes and microscopes, along with a dedicated shutter button for quickphoto capture.
Another unusual inclusion is a waterproof 4W loudspeaker, embedded in the hexagonal camera bump, that's capable of reaching up to 118dB. It’s an unusual choice for a device in this category, suggesting a shift toward entertainment and daily use, making it feel closer to a business smartphone than a traditional field device.
While not a typical productivity smartphone, the dual-screen layout and advanced sensor suite elevate its potential as a rugged tablet replacement for those who need portability with function.
The Armor 30 Pro will be available from April 14, priced at $379.99 on Aliexpress, $549.99 on the official store, €499.99 on Amazon, 33,015₽ on Ozon, and $499.99 on Mercado Livre.
Manus, the much-hyped Chinese AI, seen by some as the ‘next DeepSeek’ has opened up public access, giving you 1,000 credits to try it out for free.
Manus is the latest Chinese AI, capable of deep research and agentic usage, so it can perform whatever tasks you set while you’re free to go off and do something else. Initial users reported problems with signing up, and for most people access had been restricted.
I’d been on the waitlist since Manus was announced, and got access last Friday, but it also looks like public access is now available to anybody who downloads the iOS or Android app or signs up at its website.
Many people have described Manus as the best experience of AI they’ve had so far, and I can see why - the reports it produces are incredibly detailed and feel like they’re well-researched.
Not enough credits
Manus is not offering anything particularly new that you can’t get ChatGPT to do, but it does feel like its research is really thorough, and the range of things it can do is impressive, from making games out of current news scenarios to data analysis and education applications.
The bad news is that 1,000 credits don’t last very long, and you’ll need to sign up for a paid-for account if you want more credits.
By the time I’d got Manus to answer two queries I’d used up about 500 credits. The first question I asked (“What does the future look like for Tesla?”) was far from trivial and required a lot of research, but to its credit, Manus did all the necessary research, telling me what it was doing at every step, and produced four different reports for me.
Since DeepSeek was noted for refusing to answer questions relating to events that the Chinese government is sensitive about, I took advantage of the free access to Manus to ask it to compile a report into what happened in the Tiananmen Square protests in 1989.
DeepSeek simply refuses to acknowledge the protests, but Manus appears to have no censorship issues at all. It produced a full report into the protests from several different sources who disagree with the official verdict on things like the death toll, including the Red Cross.
(Image credit: Shutterstock/ DIA TV)
Paid-for options
So, it looks like you can get about four queries out of Manus for free with your 1,000 credits, until you have to sign up for a paid plan to use it any further.
Manus Starter costs $39 a month (about £30 / AU$65) and gives you 3,900 credits, the ability to run two tasks concurrently, while Manus Pro costs $199 a month (about £156 / AU$334) and gives you 19,000 credits a month and the ability to run five tasks simultaneously.
Port of Seattle sent out notification letters to people affected in the August 2024 ransomware attack
The letter was sent out to roughly 90,000 people
The attack was conducted by the Rhysida ransomware group
Port of Seattle, the US government agency overseeing the city's seaport and airport, has begun sent out data breach notification letters to people affected by the August 2024 ransomware attack, revealing around 90,000 were affected.
“The Port of Seattle today announced that notification letters have been mailed to individuals whose data was impacted in the August 2024 cyberattack,” the company said on its website.
“This website notice is intended to provide the same information included in the notification letters to individuals for whom the Port has insufficient or out-of-date contact information.”
TransUnion is a credit monitoring service that helps you stay on top of your financial health. With real-time alerts, credit score tracking, and identity theft protection, it ensures you never miss important changes. You'll benefit from a customizable online interface with clear insights into your credit profile. Businesses also benefit from TransUnion’s advanced risk assessment tools.
In the letter, Port of Seattle said that it suffered a cyberattack on August 24, 2024. After analyzing the attack, it discovered that the attackers “accessed and downloaded some personal information from Port systems,” including legacy systems used for employee, contractor, and parking data.
“The Port holds very little information about airport or maritime passengers, and systems processing payments were not affected,” it further explained.
The data that was taken, belonging mostly to current and former Port and other employees and contractors, included full names, dates of birth, Social Security numbers (or last four digits), driver’s license or other government ID card numbers, and medical information.
The agency notified some 90,000 individuals of the incident, most of which (around 71,000) are from Washington state.
The letter also confirms the Rhysida ransomware group was behind the attack. Even though the criminals demanded payment in exchange for releasing the systems and deleting the stolen data, the Port decided not to pay.
"We have refused to pay the ransom demanded, and as a result, the actor may respond by posting data they claim to have stolen on their darkweb site," the Port of Seattle said at the time.
Rhysida is one of the bigger ransomware operators, having assumed responsibility for the attacks on the British Library, the Chilean Army, Insomniac Games, and many others.
