you really need to start reading that chapter at verse one to get the understanding of what Paul is saying:
Galations 4: 1-11
1] Now I say, That the heir, as long as he is a child, differeth nothing from a servant, though he be lord of all;
 But is under tutors and governors until the time appointed of the father.
 Even so we, when we were children, were in bondage under the elements of the world:
 But when the fulness of the time was come, God sent forth his Son, made of a woman, made under the law,
 To redeem them that were under the law, that we might receive the adoption of sons.
 And because ye are sons, God hath sent forth the Spirit of his Son into your hearts, crying, Abba, Father.
 Wherefore thou art no more a servant, but a son; and if a son, then an heir of God through Christ.
 Howbeit then, when ye knew not God, ye did service unto them which by nature are no gods.
 But now, after that ye have known God, or rather are known of God, how turn ye again to the weak and beggarly elements, whereunto ye desire again to be in bondage?
 Ye observe days, and months, and times, and years.
 I am afraid of you, lest I have bestowed upon you labour in vain.
In verse 8, Paul told the Galations when they did not know God, they did service to them which by nature are no god. In verse 10, he told them YOU (Galations) observe times, etc…..They started doing service to the true and living God, then they turned back and started observing paganism. In verse 10, Paul is not referring to the days and months and years the Lord himself set up, but the ones they set up to their pagan gods (ie Easter, Christmas, Good Friday, Halloween, etc….). The Lord’s holy days are set up by Him in Leviticus 23rd chpt…….