The Army is feeding hundreds of hours of video recordings into a large language model to improve maintenance on the infantry squad vehicle, and hopes to eventually see soldiers using AI-powered smart glasses to fix vehicles. 

“Eventually, I’m going to run those LLMs on soldiers’ faces with smart glasses, and I’m going to run them on their phones with things like the Android [Tactical] Assault Kit. I’m going to run them on their vehicles for maintenance,” Alex Miller, the Army’s chief technology officer, said on a recent podcast with the Washington AI Network.

For the ISV, a ruggedized version of the Chevy Colorado pickup truck, Miller said, they’re training the AI with “about 1,000 hours of video recordings of our engineers, our maintenance folks, just repairing,” the vehicle. The goal is to train a visual LLM integrated with smart glasses or HoloLens that soldiers can use to “diagnose a problem” and “walk through” the steps needed to repair it. 

The Army also wants to digitize other support documents, such as log books, he said. 

“So we will have two different versions: One is Smart Lookup, which is machine learning, and one will be, actually, generative. In terms of, how do we do this? It has to run on their face, and it has to run in a way that doesn’t get so hot that we burn our soldiers,” like when using a phone in direct sunlight causes it to overheat.  

Miller’s comments come on the heels of the release of  a White House AI Action Plan to accelerate military adoption of AI, and as the Pentagon has quickly embraced generative AI with multiple $200 million contracts to assist with back-office tasks.  

The Army has been experimenting with generative AI for daily operations ,and has considered using AI for more dangerous missions like disarming bombs and air operations, looking at ways to bring AI tools closer to the battlefield and in smaller formats, like laptops or handheld devices. The service also has been investing in AI on the enterprise level, with a potential $10 billion enterprise contract with Palantir and a $99.5 million research and development contract with Scale AI announced Thursday. 

Additionally, the Army is working with several companies to “look at boundary conditions” and see how AI can be best used to scope out potential threats, Miller said.  

“If you look at what a lot of the language domain models and LLMs do, they can do some pretty spooky stuff—from new chemistries to new material sciences. We want to have access to those to look at what the threats are. So, what’s in the realm of medical technology that we should be aware of? Or bio sciences or bioterrorism?” he said. 

“A lot of companies stand up venture arms, which is very cool, but they still have research arms. And they want…data back from us because we’re willing to try really weird cases all the time for them.”