modern_hopfield

torchhd.memory.modern_hopfield(query: Tensor, memory: Tensor) → Tensor[source]

Modern Hopfield network

Also known as Dense Associative Memory.

Parameters:

  • query (Tensor) – The query vector for the memory lookup.

  • memory (Tensor) – The items of memory for the memory lookup.

Shapes:

  • Query: \((*, d)\)

  • Memory: \((n, d)\)

  • Result: \((*, d)\)

Examples::
>>> items = torchhd.random(6, 512)
>>> read = memory.dense_associative(items, items).sign()
>>> torchhd.cosine_similarity(read, items)
tensor([[ 1.0000,  0.0469, -0.0117,  0.0039, -0.0313, -0.0078],
        [ 0.0469,  1.0000, -0.0352, -0.0039, -0.0391, -0.0078],
        [-0.0117, -0.0352,  1.0000,  0.0547,  0.0742, -0.0352],
        [ 0.0039, -0.0039,  0.0547,  1.0000,  0.0273,  0.0117],
        [-0.0313, -0.0391,  0.0742,  0.0273,  1.0000, -0.0547],
        [-0.0078, -0.0078, -0.0352,  0.0117, -0.0547,  1.0000]])