Read between the languages.
Read the original and translation paragraph by paragraph. arXiv papers use the official HTML version for sharper formulas and figures, and the original PDF layout is one click away.
YOUR NEXT IDEA STARTS HERE
Original paper. Translation under every paragraph. Answers you can trace. Meet the research companion that keeps you in the flow.Your AI paper reader for Mac. Translate side by side, ask with citations, and keep Markdown notes.
For Mac · Install and read on your Mac
View full screenshotSTAY WITH THE PAPER
Follow the argument, check an equation, capture an idea. Keep the context while you do it.
Read the original and translation paragraph by paragraph. arXiv papers use the official HTML version for sharper formulas and figures, and the original PDF layout is one click away.
The Transformer uses attention to connect relevant parts of a sequence, without recurrence.
Abstract · p. 1 ↗Illustrative answerAsk about a passage or a screenshot. Follow paragraph citations back to the paper and check the reasoning yourself.
Compare attention patterns across longer sequences.
Highlight a passage and add a note. Notes are plain Markdown files you can open in your own editor.
SAME PAPER. A DIFFERENT WAY IN.
Some days you want the details. Some days you just want someone to explain it like you’re eight. Pick your way in.
Swipe to explore reading modes
Think of attention as looking something up in a filing cabinet. You come with a question, and each folder has a label on the front and papers inside. You compare your question to every label to see how well it matches, and then you build your answer by mixing the papers together — taking a lot from the folders whose labels matched well and only a little from the others§3.2.
In this paper, the matching is done by a simple comparison of the question with each label, and then the match scores are shrunk a bit before they are turned into the mixing amounts§3.2.1. That shrinking matters: without it, when the labels are long and detailed, the scores get so big that the whole thing becomes hard to train.
They also do this several times side by side, each copy looking at things in its own way, and then join the results — so the model can pay attention to different kinds of information at once§3.2.2.
Think of attention like looking things up in a lost-and-found box. You bring a question, called a query. The box holds labels, called keys. Each label sits next to an item, called a value. The computer checks how well your question matches each label. Then it mixes the items together, giving more of the ones that matched best. That mix is the answer§3.2.
The Transformer does this with multiplication. It multiplies the query by every key. Then it divides by the square root of the key size. Then softmax turns those numbers into weights§3.2.1. Why divide? Without it, big numbers make learning almost stop, because the gradients get very tiny. That is bad, so the division helps a lot.
One last trick. The model does this many times at once, in parallel heads. Each head looks at a different kind of clue. Then the answers are joined together§3.2.2.
Attention starts from three kinds of vectors: a query, and a set of key–value pairs. Each key is compared to the query by a "compatibility function," which produces a weight for the value paired with that key; the output is then the weighted sum of the values§3.2. So the query asks "what do I need?", the keys advertise what each position offers, and the values carry the content actually passed on.
In this paper the comparison is a dot product. For a query and keys of dimension d_k, and values of dimension d_v, you take the dot product of the query with every key, divide each by √d_k, and apply softmax to turn those scores into weights§3.2.1. Batched, queries stack into matrix Q, keys into K, values into V, giving Attention(Q,K,V) = softmax(QK^T/√d_k)V. The √d_k division matters because large d_k makes dot products large, pushing softmax into flat, tiny-gradient regions.
Multi-head attention repeats this h times on different learned linear projections, then concatenates and projects the results, letting the model attend in several representation subspaces at once§3.2.2.
Master three things. Attention maps a query and a set of key–value pairs to an output that is a weighted sum of the values, the weights coming from a compatibility function between query and key§3.2. The paper's version is scaled dot-product attention: dot the query with all keys, divide by √dk, softmax, multiply by V, computed in batch as softmax(QK^T/√dk)V§3.2.1. Multi-head attention projects Q, K, V h times into lower dimensions, runs attention in parallel, concatenates and projects, letting the model attend to different representation subspaces at once§3.2.2.
Expect these questions. Why divide by √dk? Because for large dk the dot products grow large and push softmax into low-gradient regions; unscaled dot-product attention then loses to additive attention§3.2.1. Why abandon recurrence? The Transformer relies solely on attention, is more parallelizable, trains faster, and reaches 28.4 BLEU on WMT 2014 English–German§Abstract.
If asked about positional encoding, masking, complexity per layer, or head counts, say plainly that these excerpts do not cover them.
Mechanically, attention maps a query against a set of key–value pairs and returns a weighted sum of the values, the weights coming from a compatibility function between query and key§3.2; here that function is a dot product divided by √dk, softmaxed, and computed batched as matrices, softmax(QK^T/√dk)V§3.2.1. Multiple such functions run in parallel on learned linear projections of Q, K, V, and their outputs are concatenated and projected once more§3.2.2.
Now the gaps. The scaling factor rests on a hunch, not evidence: "we suspect that for large values of dk, the dot products grow large" and push softmax into small-gradient regions§3.2.1 — no measurement, and the accompanying claim that additive attention beats unscaled dot-product at large dk is stated without any experiment in these excerpts. Likewise "we found it beneficial"§3.2.2 is asserted with no ablation numbers, and "different representation subspaces" is interpretation, not demonstration.
The excerpts also never give h, the relation between dk and dmodel, masking, or the cost of the full QK^T — so the quadratic-in-sequence-length price of "dispensing with recurrence"§Abstract goes unquantified here. Finally, "superior in quality" on "two machine translation tasks" is supported by one reported number, 28.4 BLEU, with unnamed baselines.
YOUR RESEARCH. YOUR SETUP.
Connect your own API key or a model running on your own computer. Choose models by task and keep an eye on estimated usage.
PDFs and notes are stored locally. Cloud AI sends the content needed for your request to the service you configure.
LESS FILE HUNTING
Find papers on arXiv, Semantic Scholar and OpenAlex and add them to your library. Titles are translated, authors fill in automatically, and pins and tags keep things sorted. Pick up where you left off.
GET READARC
A one-time purchase on the Mac App Store. No subscriptions, no bundled token markup.
The ready-to-use macOS app, with installation and updates through the App Store.
macOS is available now on the Mac App Store. A Windows version is in development.
FAQ
ReadArc uses API access from the provider you choose, or a local model. Cloud API usage is billed separately by that provider; a chat-app subscription may not include API access.
Yes. Imported PDFs, saved translations, the bundled dictionary and notes work offline. Cloud AI and online paper search need a connection.
No. Your library is stored on your computer. When you use a cloud AI feature, the content needed for that task is sent to your configured endpoint. A local gateway may also forward requests to the cloud.
The Mac App Store version is a one-time purchase: CNY 18 in China, US$2.99 in the US. Other regions follow the store price, with no subscriptions or in-app purchases. Cloud model API usage is billed separately by your chosen provider.
macOS is available now on the Mac App Store. A Windows version is in development.
READ. QUESTION. CONNECT.
Give it a little attention.
Get it on the Mac App StoreGet the Mac app