The women, mostly in their 70s, said that their age and gender made them particularly vulnerable to the effects of heatwaves linked to climate change.
The court said Switzerland's efforts to meet its emission reduction targets had been woefully inadequate.
It is the first time the powerful court has ruled on global warming.
Swedish campaigner Greta Thunberg joined activists celebrating at the court in Strasbourg on Tuesday.
"We still can't really believe it. We keep asking our lawyers, 'is that right?' Rosemarie Wydler-Walti, one of the leaders of the Swiss women, told Reuters news agency. "And they tell us it's the most you could have had. The biggest victory possible."
"This is only the beginning of climate litigation," said Ms Thunberg. "This means that we have to fight even more, since this is only the beginning. Because in a climate emergency, everything is at stake."
The ruling is binding and can trickle down to influence the law in 46 countries in Europe including the UK.
The Court ruled that Switzerland had "failed to comply with its duties under the Convention concerning climate change" and that it had violated the right to respect for private and family life.
It also found that "there had been critical gaps" in the country's policies to tackle climate change including failing to quantify reductions in greenhouse gases - those gases that warm Earth's atmosphere when we burn fossil fuels like oil, coal and gas.
The Swiss women, called KlimaSeniorinnen or Senior Women for Climate Protection, argued that they cannot leave their homes and suffer health attacks during heatwaves in Switzerland.
On Tuesday data showed that last month was the world's warmest March on record, meaning the temperature records have broken ten months in a row.
More than 2,000 women are in the KlimaSeniorinnen group. They launched the case nine years ago, calling for better protection of women's health in relation to climate change.
Swiss President Viola Amherd told a news conference that she needed to read the judgement in detail before commenting, according to Reuters news agency.
She said: "Sustainability is very important to Switzerland, biodiversity is very important to Switzerland, the net zero target is very important to Switzerland."
The court dismissed two other cases brought by six Portuguese young people and a former French mayor. Both argued that European governments had failed to tackle climate change quickly enough, violating their rights.
Member of the KlimaSeniorinnen Elisabeth Stern, 76, told BBC News that she has seen how the climate in Switzerland has changed since she was a child growing up on a farm.
'Not made to sit in a rocking chair and knit'
Asked about her commitment to the case, she said: "Some of us are just made that way. We are not made to sit in a rocking chair and knit."
"We know statistically that in 10 years we will be gone. So whatever we do now, we are not doing for ourselves, but for the sake of our children and our children's children," she added.
Youth activists around the world had hoped that the six Portuguese young people would also win their case against 32 European governments.
The youth, aged from 12-24, had argued that increasingly extreme heatwaves and wildfires left them unable to go outside to play, to go to school, and that they suffered from climate anxiety.
But the court said the case needed to be decided in Portugal first.
Sofia Oliveira, 19, told BBC News that she was disappointed but that the Swiss women's win 'is a win for us too and a win for everyone'."
A third case by a former French mayor claimed that inaction by the French government risked his town being submerged in the North Sea.
It was dismissed because he no longer lived in France and claimants must prove that they are direct victims of human rights violations.
Decisions made in the European Court of Human Rights influence law across its 46 member states.
Estelle Dehon KC, a barrister at Cornerstone Barristers in the UK, said "the judgement deals with difficult issues that also vex the UK courts in a way that may be persuasive."
"It comprehensively dismisses the argument that courts cannot rule on climate legal obligations because climate change is a global phenomenon or because action by one state is just a 'drop in the ocean'," she told BBC News.
Governments globally have signed up to drastically reduce their greenhouse gas emissions.
But scientists and activists say that progress is too slow and the world is not on track to meet the crucial target of limiting global temperature rise to 1.5C.
Switzerland's largest party, the right-wing Swiss People's Party, condemned the ruling, calling it a scandal and threatening to leave the Council of Europe.
That is unlikely to happen because they hold just two seats of seven in government.
The Socialist Party welcomed the court's decision and said the government should implement it as soon as possible, according to Swiss broadcaster RTS.
Politicians in Switzerland responded to the ruling
By Georgina Rannard