Because the peel is designed to protect the inside of the orange, its structure is different. Besides being tougher than the sweet interior, the peel also contains a lot of air. The peel is less dense than water or light enough to float, but the inside Fruit is denser than water so it sinks.
Different Fruits and Vegetables will also float or sink depending on their density. In general, apples, bananas, lemons, oranges, pears, and zucchinis will float, while avocados, potatoes, and mangoes will sink. Others like turnips and sweet potatoes sometimes sink and sometimes float.