You will need a wire brush to get the rust off. It’s a lot to do overhead by hand. You can get a wire knot brush attachment for a grinder.
Use a high quality rust primer before painting.
If you want to match the downpipes compare paint chips or unscrew a bracket and take it to paint shop for colour matching

It is always advisable to attempt to match the down pipes to the wall colour. An opinion only so disregard if it is not warranted. This pinky colour on your down pipes is very dated. Depending on the colours you choose, you might consider another colour that will go with the walls and your new paint choices. An option could be the lighter brown in the darker bricks or if you choose a dark colour paint, try it with your reddy bricks.

Regarding the COLOUR of painting all the trims. I recommend picking up a colour from the mortar in the bricks. Painting all the carport, windows, gutters, fascia, and down pipes in that same colour. The existing red colour on the down pipes to match the bricks is dating the exterior of your home. You could also paint all the trims, carport, windows, gutters, fascia and down pipes in a dark colour the same as the flecks in the brick. But the mortar is very light, so that is why I would look at a grey colour first. Examples of colours in attached photos.

This house is probably not the era of yours. But it just shows how are deep grey can look wonderful against a red brick with light mortar.

Thank you very much, everyone. (Not sure if I should ask this in another post)My gutters are green in colour, and the gables are green and white. Do you think I should consider a different colour scheme? My pergola too is green colour.

Refer to my post above. Do all the trims in a dark grey. That means gutters, fascia, down pipes, gables and your pergola etc etc

Personally I would paint all the green in a dark charcoal. Have a look at Dulux Namadji which has a slight brownish tinge to the charcoal base  It would go well with your bricks and roof. (I don't  know if you would be open to removing the decorative finials from your gables which are very nineties and dated.)

Thank you very much. Do you have any advice on how to remove oxide from gutters and fascia ( 1st photo )? Can it be chemically removed or should I do it manually with a sanding brick? It would require significant effort to do it manually as it is around the entire house.

The oxide is actually pitting and rust bubbles. It needs to be sanded back and coated with the appropriate primer (available from
Bunnings or your local paint shop). You could probably paint over it, the primer would stop further damage but it won’t be smooth nor look good. A detail sander is easier to hold whilst up a ladder but may not have the oomph required. I’d probably use a wire brush on a drill then a belt sander but if you go to Bunnings outside of busy times and show them the pic, they’ll help you. AND don’t pick up a paint brush and paint anything until you’ve decided the colour scheme for the entire house.

Thank you everyone for .your valuable input.