Be sure to scroll down... there may be more than one question on this page!
The plastic anchors work just fine as long as they fit tightly into the holes. Because the anchors act as a wedge to hold the screws into the very hard cement or brick, you can use a larger screw than you would use if installing the same anchor in drywall. In other words, if you are using a #6-8 anchor, use either an #8 or #10 screw. Though the anchor may crack, it will wedge within the cement so tightly that it will be virtually impossible to pull out!
There are special purpose anchors designed to be used in masonry, made from lead and steel, but for all but the most heavy duty jobs (such as cabinets) the plastic anchors will work just fine. In fact, even a tightly fitted wood dowel would expand enough to firmly hold a screw in masonry, provided the dowel is not too much larger than the screw. Again, the principle is that the screw when installed will expand the anchoring medium so that it is tightly wedged against the sides of the hole in the masonry.
As an aside, this is why plastic expansion anchors do not work well in drywall. Drywall is fairly soft, so that the anchor cannot wedge tightly against it. Instead, the drywall deforms slightly, providing only minimal holding strength.
Back to your question, you may drill into either the brick or the mortar. However, I prefer to fasten to the mortar for cosmetic reasons… it is easier to touchup holes in the mortar if you someday take down the mantle. You cannot do too much to disguise the holes in the brick face unless it is painted or you are very artistic. In that case the holes can be filled with virtually any filler suitable for wood or cement prior to being touched up with paint.