Mortgage rates that have doubled over the past year are sapping demand from the housing market, which saw prices decline for the third month in September.