Instances of OpenLayers.Marker are a combination of a OpenLayers.LonLat and an OpenLayers.Icon.
Markers are generally added to a special layer called OpenLayers.Layer.Markers.
Example
var markers = new OpenLayers.Layer.Markers( "Markers" );
map.addLayer(markers);
var size = new OpenLayers.Size(21,25);
var offset = new OpenLayers.Pixel(-(size.w/2), -size.h);
var icon = new OpenLayers.Icon('http://www.openlayers.org/dev/img/marker.png', size, offset);
markers.addMarker(new OpenLayers.Marker(new OpenLayers.LonLat(0,0),icon));
markers.addMarker(new OpenLayers.Marker(new OpenLayers.LonLat(0,0),icon.clone()));
Note that if you pass an icon into the Marker constructor, it will take that icon and use it. This
means that you should not share icons between markers -- you use them once, but you should clone() for any
additional markers using that same icon.