I to did the same and just took them out, it dropped the front almost 2" from the stock HD 4x4 height, I was having problems with the tierods bending on 4x4 launches and this fixed it.
Even with the bolts all the way out there is still alot of preload on the keys so there isnt a chance of a bar slipping out at full droop.