Thinly slice garlic. Peel 4 thick strips from orange skin. Carefully cut away white pith. Slice peel into thin strips. Pour oil into a large frying pan set over medium heat. Add garlic, orange strips, rosemary, cumin and chili flakes. Stir often until fragrant, 3 min.
Add olives and stir until warm, 2 min. Turn into a serving bowl. Discard rosemary. Serve warm or leave at room temperature up to a day or refrigerate up to 2 days. Reheat in microwave, if you like. After olives are eaten, use crusty bread chunks to sop up oil.