As far as I know its supposed to hold when key off. Mine is doing the same thing now. Wasn't before when I first swappedthe motor in. I changed regulator and no fix. Put new injectors but cant test yet. Others have had same issue and chances are its the hose in the tank between the pump and unit. Ill be checking that soon. I suggest u do too.

Yes it will run fine as is but at WOT at high rpm, motor will bog. If ur under boost, ull kill the motor. just cuz it runs doesn't mean its ok.