Hilchos Issurei Mizbeiach 6:1

Just like there’s a mitzvah for offerings to be unblemished and choice, their accoutrements must be unblemished and choice. Numbers 28:31 specifies “They shall be perfect for you – they and their libations.” We see that the things that accompany the offerings must also be unblemished. One may not bring libations of smoked wine or wormy flour, nor may one mix flour with oil that smells or tastes bad.

Hilchos Issurei Mizbeiach 6:2

Wood for the woodpile must also be of the best quality, and not wormy. Any tree that is infested with worms when fresh is unfit for the altar. If the wood became infested with worms after it was dried, the infested spot may be scraped off. Wood from a building that was razed is unfit; only new wood may be used.