Initial Charge Voltage For A 12 Volt Lead-Acid Battery is 14.4 Volts

After charging up fully the regulator drops the voltage down to around 13.6 volts

What about lithium phosphate batteries?

A charge voltage between 13.8 and 14.7 volts will charge a 12 volt LiFeP04 battery.

Is It OK To Use A Very Small Solar Panel?

Yes, if the maximum current output is lower than the float charge current for the battery. In this case a charge regulator is not required