The answer is simple: yes, but it is not easy. The LFP lithium battery is made of materials that are flammable and it will burn, if overheated.
The photos show the process of fire on a prismatic lithium battery. The battery was fully charged and then set to fire by a flame-thrower. As you can see the battery material burn slowly.
The result is: there was no explosion, there was no burn-out or no massive flame burst reaction. The fire was slow and gradual.
Actually, as seen in this video, the battery had to be set to fire repeatedly (!) before it heated up enough to start burning by itself.