There are some ways that you can make going out into the cold more comfortable for your puppy and help train your Boxer to go outside in the cold for housebreaking:
Put a doggie sweater, vest or coat on your puppy. Clothes for dogs is not just a method of making a dog look "cute". Dog clothes really do protect a dog, especially a puppy from the harsh elements outside and a doggie sweater will certainly help keep you pup warmer.
When your Boxer is led outside with an outer layer of protection, and realizes that the cold air is not biting at him, he will be more prone to actually enjoy being outside, exploring around a bit and relieving himself.
You may also want to consider doggie boots. Again, these are not just a way of "dressing up a dog". A dog's paw is made of skin. While it is thick skin, it is still skin. It is a very huge misconception that a dog's paw can handle any surfaces. A puppy will feel the cold on his paws.
Protecting those puppy paws with boots or dog shoes will allow your Boxer to walk around in comfort and not feel the discomfort of the near freezing temperatures.
When humans go outside in the winter, we do not just wear shoes, we wear 2 layers of socks and heavy snow boot; therefore a Boxer puppy or dog needs at least a bit of protection aside from the thicker skin of their paw. Another option is to lather the paws with a quality paw wax two to three times per week.