Could be you polished it too soon. I've had "dieback" because the clear was not fully cured when polished. Could be you didn't really didn't get all the "scratches" from the sanding operation out when buffing the first time. Or, you didn't get all the 1000 grit scratches out when doing the 1500 grit sanding. Then the compound masked the problem. I've had this problem with the modern compounds, they seem to fill and create and artificial shine that doesn't last. I even asked advice on which compounds did not do this, on this forum.
If the paint has acquired a texture, as you say, it probably wasn't cured enough, when you sanded and buffed, and kept shrinking afterwards. Since I do mostly cars, long term projects, I can afford (time-wise) to wait a month before sanding and buffing, and I don't have this problem. But it DOES make sanding and buffing a much harder job.
