If your amp's output impedance is 4 ohms and you put and 8 ohm speaker on it, you will have 1/2 the power out of the speaker at a given setting of the volume control.
However, that may not be as bad as you think, since it takes a power change of 10 times in order for the ear to perceive a change of twice (or 1/2) the power. In actuality, you may notice no difference, even though, your speaker is only being fed half the power. And probably NO loss in fidelity in any case, everything else being equal.
Of course speakers of different make and style can make a decided change in perceived sound, EVEN if the amp's output impedance and speaker's impedance are identical.
In otherwords different speakees of different style and make can make a much greater difference in sound than the same make and style speakers of different impedances.
Charlie, give yourself a real treat and use 2- 8 ohm speakers. You will acheive a full load like this with the advantage of stereo sound. One speaker in the amp and one extension.
Dave