Heathens are people who were made to seek out places to live on The Heaths...swampy marshy lands that had no value because they were horrible places to live. They were forced there when they had their lands taken from them by noblemen working for The Church. It has come to mean a person who goes against the doctrines of the Christian Church. The Church would very often arrest and execute people who wrote literature, painted artworks, or indulged in any artform that they thought did not agree with what they thought people should know about. Anyone trying to embellish peoples knowledge of life via creative works would be deemed heretic and their works would be destroyed.
So Bowie crossed his own work out because he is seen as a Heathen. A denial of Christianity or at least the Church's version of Christianity would indeed make him a Heathen in their eyes.
The symbol of the fish can apparently been seen in Bowie's eyes on the front cover of Heathen. This symbol is a very well known Christian symbol that denotes your devotion as a Christian to Jesus. It is used because there are a few references in the Bible involving Fish/Fishing/Fishermen that Jesus concerned himself with. Christians would use the symbol to determine whether others were Christian. If they drew one side of the symbol, the other Christian would draw the other on greeting them. This was in times of persecution.
However... I am wondering whether Bowie is using the Pre-Christian use of the symbol. It goes back before Jesus and has been a symbol used in mystery teachings from various religions including, India, China, Egypt, Rome, Scandinavia and Greece. It's usually associated with the Mother Goddess symbolising her vulva. Hence in this respect it would be a Heathen symbol rather than a Christian one. As Bowie seems to be aligning himself with Heathens then this is possibly how he is using the fish symbol. So it is indeed a nice ironic statement.
(Although he may be suggesting he is a Heathen, this does not mean he is Athiest. He is Heathen in the eyes of The Church).
David has mentioned in an interview that he chose the three books depicted in the album sleeve because these books totally changed our attitude to life and our world view in the 20th century.
The three books depicted in the album sleeve are: The General Theory Of Relativity - Albert Einstein, The Interpretation of Dreams - Sigmund Freud and The Gay Science - Friedrich Nietzsche.....
These three books propelled human thinking into the realm of humanism. They made people start to question whether their existence on this Earth is not a spiritual one but rather a very random and inconsequential one and whether we should all just take care of ourselves on a physical level instead of striving for something bigger on a spiritual level. These books helped to bring in the age of reason on a grand scale and put a halt to the idea that there is something bigger out there for us to aspire to spiritually. (I think he should have included writings by Rene Descartes too, who also has a lot to answer for in the disintegration of spiritualism). If we use empirical thought processes instead of gnostic searching for answers then we come to the conclusion that everything is physical and finite. ( I don't subscribe to this).
But I think Bowie was questioning whether he should subscribe to these viewpoints, effectively turning his back on religion and more alarmingly, spirituality itself. This too would make him a Heathen in the eyes of The Church. Note, he is probably only wondering whether he should take on the stance of the three authors in question, there is no proof that he has yet. (Back to that Atheist thread...)
Bowie desperately wants to be an Atheist but he can't quite bring himself to do it. If you notice from his last album to date written 5 years ago now, Reality has him writing about his constant searching for answers and never actually finding any. It shows he has become cynical and skeptical in his older years due to a process of too much self-questioning. Soul searching is very painful but being Atheist is quite peaceful for the soul:
Never Get Old:Putting on my gloves and I bury my bones in the Marshland
Marshland- like the Heaths that Heathens had to live in.
He's decided that's where his viewpoint lies.
Think about my soul but I don't need a thing, just the ring of the bell in the pure clean air
He's questioned his spirituality but has decided that there's nothing more he needs to answer because he just needs to think about the now and his physical life here and how to be happy without questioning.
New Killer Star: All My idiot questions, let's face the music and dance
He's sick of searching for answers and has decided that there's no answers to give to his questions anyway so let's just face it, there's nothing more to life than what we have on this planet in the present. Let's just have fun now and stop searching.
The Loneliest Guy: No Sign, no nothing now, but I'm the luckiest guy, not the loneliest guy
He's not found any answers to his spiritual questions, he's had no sign that's there anything more to life. But he's come to accept this and can count himself lucky that he's living life peacefully without the searching.... because searching is painful and lonely.
All the pages left unturned, all the errors left unlearned.
He has not found his answers in books and cannot learn from his mistakes after so much searching and questioning.
Looking For Water: But I lost God in a New York Minute, I don't know about you but my heart's not in it. I'm looking for water.
He's lost the will to carry on with his spiritual search and his own ideas about God have left him bitter, leading him to Atheism. Yet he's still looking for something..something as intangible as flowing water.
Leave it all behind, nothing will be found
Just stop searching you won't find any answers.
Looking everywhere, looking here and there. I'm looking for water.
Looking for the answers to life and the soul is like looking for something intangible.
I can't live in this cage, I can't eat this candy. To the ends of the Earth, to this pain in my head...
It's too difficult to keep questioning his soul, it's painful searching all the time and tiresome trying to get answers to the many thoughts in his head.
Baby dumb is forever.
Just stop questioning, you'll never get the answers and you'll always stay ignorant no matter how hard you try.
Days: In red eyed pain I'm knocking on you door again, this crazy brain in tangles, pleading for your gentle voice.
These storms keep pounding through my head and heart. I pray you'll soothe my sorry soul.
Here he is possibly talking to his God again, he is desperate to believe in something. He has so many thoughts and questions and wants answers so the pain of searching will go away. He wants his God to talk to him again so he doesn't lose faith.
Reality: Now my sight is failing in this twilight, now my death is more than just a sad song
He's getting older and hasn't got many more years left for this earth. He's losing the ability to see clearly about his faith in something bigger. Becoming Atheist.
I still don't remember how this happened, still don't get the wherefores and the whys.
He can't understand how he's lost his faith through all the searching.
I look for sense but I get next to nothing. Welcome to reality.
He keeps trying to makes sense of life and why we're here, he keeps asking questions and searching but still he never finds the answers. He's decided there are no answers because there is nothing more out there and no God as such. This is the reality of it all.
I've been right and I've been wrong, now I'm back where I started from.
He has come close to finding what he thought were answers to his questions about the soul and life then he's decided that those answers weren't right. And after all that questioning through his life, he's still no wiser.
With Heathen, he questioned whether he should turn his back on religion and spiritual soul searching and with Reality he answers his own question. He essentially says he has. But with the song Days we can see that he still wants answers and still wants to believe...but too much of this questioning is hard to live by. Just live life and forget about the answers, it's easier that way.
And so this has become his last album... he has no more questions and nothing left to write about. He's getting older and facing his own mortality and has decided to live his life as peacefully as he can with his family and reap the rewards he so deserves. If he decides to start questioning again...then perhaps we'll see a new album.
And there's only so many songs you can write about New York anyway.