Waddup! To me it sounds like you may not be stretching the strings out properly after installing them and before attempting to set up/calibrate your trem for playing.
Since you mentioned you're new to a floyd rose system i will list below my method that works for me! I always clean my guitar as well when changing strings however i will leave those details out and focus on what to do for the tremolo.
- unscrew and remove locking nut screws @ head of guitar
- unwind all tuners allowing tremolo to "deck" or rest against bottom of trem cavity.
- with the trem arm still installed into the base plate, press down (like you're doing a dive) to allow access to the string holder locking bolt to remove strings from tremolo.
- cut the ball ends off of all new strings
- press down on trem arm again and install new strings into saddle making sure they are inserted all the way into saddle and centered, then tighten string holder locking bolt making sure not to go crazy tight as you can pinch the string and cause premature breakage.
- then begin to wind each new string into tuners starting with low to high strings. Dont worry about tuning the strings yet, just focus on winding each string up enough to notice the tremolo bridge starting to raise up again towards its usual position.
- after all strings have been installed and the tremolo is no longer "decked" start stretching the strings one at a time with your hand. place your thumb on the string and use your other fingers to press the string against your thumb/palm as if you're almost pinching the string. do this up and down the string every so often from the bridge all the way up to the fifth fret or so. (that was alot harder than i thought to describe so also like this guy is doing)
- then tune each string (alternating from low to high...low-e, high-e, a, b, d, g) and start playing and using the tremolo to stretch the strings out even more. be patient cause you will have to repeat this step atleast a dozen times lol
- once the tuning has stabilized take note of where your tremolo base plate is. you may need to adjust the claw screws under the dust plate and retune to make the base plate parallel with the guitar body.
- install locking nut screws again and you're ready to rock!
You can raise and lower string/bridge height by adjusting the (2) studs that the tremolo base plate rests against.
NEVER ADJUST THESE STUDS WHILE THE GUITAR IS STRUNG. The two cut outs in the trem base plate have a knife edge, this is where the base plate pivots off the studs and if you adjust the stud while the guitar is strung you risk flattening the knife edge, if you have a flat spot you will notice a subtle imbalance or knock when using the trem.
You will have to play around with the bridge/string height yourself to see what feels the best. I feel like lower action is most popular however you have to be careful not to go too low to avoid string buzzing. There is alot of info out there on suggested string heights that you can go off of.
You can refer to this
diagram to better describe the terminology of floyd.
Let me know what you think or if this was confusing and i can try to better help