Leon, my guess is that your lift pump is starting to fail. My truck did the exact same thing before the lift pump died completely. The dealer now has an in tank pump retrofit that they do for our trucks. This is all based on the assumption that you still have the stock engine mounted lift pump... is that correct? If so, my guess is that the LP is going south.
2004.5 2500 Quad Cab LB 4x4 with stuff.