Suggest taking 危 as person on a cliff, in danger of falling off and breaking his body (肉 / ⺼).
Mnemonic: If you fall of a dangerous cliff you break easily and become dead meat
According to Google moroi is written about as often in kana もろい as in kanji 脆い.¹ Till now I myself had not seen 脆 and in fact have still to encounter it by simply reading Japanese (I did come across もろい). I bumped into 脆 reading a line from Laozi 老子.
其脆易泮 What still is brittle can be broken easily.
So what is the deal with 脆 ? On first glance it's simply 肉 / ⺼ nikuzuki as signific and 危 as phonetic. And if we want to imagine that 危 has not only phonetic value, but also semantic value, that's easy since 危 depicts a cliff with a person crouching on its edge in danger (危ない) of falling and breaking his bones → easily broken, brittle.
But why flesh/meat as signific? What has that to do with ‘crisp; fragile, frail; brittle’ (meanings from the Unihan data). With this signific I'd expect it to refer to people, but for breaking bones at least, 骨 would then have been a more suitable signific. However, in Japanese, moroi as a simple adjective seems mostly to refer to brittle things. When it does refer to people, some aspect is added, like ‘emotionally brittle’ as in 情にもろい jou ni moroi (soft-hearted), or ‘bursts into tears easily’ in 涙もろい namida moroi. Still, who knows how the word 脆 used to stand for started out. Maybe it did originally refer to weak bodied people.
Interestingly 脆 has a so called semantic variant: 脃, which seems to use 色 as phonetic. Note that originally 色 and 危 were almost identical in shape, only the line of the cliff in between made them different. Unihan data lists another meaning for 脃 ‘gland’ - that would make more sense with the signific for flesh/meat. However, I'm not so sure the Unihan data is very reliable for old meanings. Maybe it just lists the most commonly ascribed meanings.
Lets check the classical dictionaries at the Chinese Text Project (and ignore that it also lists the Unihan data meanings - which are so much more attractive since they are in English). It seems to me (having had no lessons in classical Chinese) that here 脃 is the original, and 脆 the corruption.
(康熙字典) 脆： ... 俗脃字。 Isn't this saying about 脆 that it is a corruption of 脃?
(說文解字) 脃：小耎易斷也。从肉，从絕省。 This is the ‘breaks easily’ meaning, nothing about a ‘gland’. Note also that in the original 說文解字 the right sides of 脃 and 絕 are still identical, unlike the versions we ended up with: compare in the facsimile edition 脃 and 絕.
So, the (older, 2nd century) 說文解字 doesn't even list 脆, and has the ‘breaks easily’ meaning for 脃, while even the 18th century 康熙字典 still concedes that 脆 is actually a corrupted form. Belatedly I also checked a modern Japanese character dictionary, and it confirms what I seem to read in the last bit of the 說文解字 entry: the phonetic is an abbreviation for 絕/絶.³ Nothing to do with 危 !
The semantic value of 絕/絶, which includes ‘sever; cut off’ is even more obviously relevant than 危 would have been, but it's hidden between two layers: 色 is an abbreviation for 絕/絶, and later on 色 was corrupted to 危. As so often with the etymology of Chinese characters, it's a mess.
Notes
