Cover potatoes with salted water in a medium saucepan. Boil just until fork tender, 9 to 12 min. Drain and cool for 5 min. Slice in half, lengthwise. Using a melon baller, scoop away small amount of flesh from centre of potato halves and discard. Sprinkle with salt and pepper.
Cook bacon in large frying pan over medium, flipping halfway, until crisp, 5 to 8 min. Transfer bacon to paper towel. Crumble. Add rosemary to pan and add potatoes, cut-side down. Cook until golden and crisp, about 4 min. Transfer, cut-side up, to platter.
Spoon 1 heaping tsp Boursin® onto centre of each potato. Top with bacon bits and chives, or caviar.