Caroline joined the Conservative Primrose League, but while working as a teacher in Reading she lodged with her maternal aunt, Mrs Annie Bailey, who held strong left-wing views. She briefly became a radical and then a socialist. In 1891, she was appointed a governess at the Royal Orphanage Asylum in Wandsworth, London, and joined the London Fabian Society.
The following year, ill-health forced her to give up work and she began to devote herself full-time to the socialist cause. However, this was tempered by the devout religious views she had inherited from her parents, and she strongly disagreed with the Marxist principles of many of her contemporaries. She gained a reputation as one of the foremost social orators of her day, large crowds turning up to hear her speak as she travelled the country. The Sheffield & Rotherham Independent, described her as ‘one of the most indefatigable lecturers of the Independent Labour Party.’ The Aberdeen Weekly journal considered Caroline to be ‘endowed with rare intelligence matured by a liberal education and close study’ and ‘one of the few female public speakers whose enthusiasm for their subjects is such that they seem to drop their identity for the time being and become thrilling mouthpieces for the arguments they adduce.’
For a while, in 1893, she was a sub-editor on the Christian Weekly.
Although she had many articles published in journals, Martyn was predominantly known as a lecturer. She became nationally recognised and large crowds turned up to hear her speak as she travelled round the country, predominantly in Lancashire and Scotland. In 1896, she was elected to the National Administrative Council of the Independent Labour Party and became editor of Fraternity, the journal of the International Society for the Brotherhood of Man, and ILP trades union organiser for the North of Scotland. Due to her rapid rise in the ILP many spoke of her as a future leader of the Party, even though Keir Hardie called her a middle-class woman.
The work and travel, however, undermined her already fragile health, Caroline had arrived in Dundee on the 13th July 1896 to encourage female workers to join the Dundee Textile Workers Union. She collapsed after a meeting on the 22nd July, a telegram was sent to her mother to go to Dundee. Caroline died of pleurisy in the late evening of the 23rd July at 107 Murraygate. She was buried in Balgay Cemetery in the city on the 24th July, her funeral was attended by her mother and her brother.
Keir Hardie wrote that she was the leading socialist of her day, with 'a power of intellect and moral-force'