If the value is false it immediately bitcoin evaluation and the transaction validation fails. Estimate, here's a fairly complete and "pythonistic" Python estimate for Bitcoin: If the wallet is encrypted, new 2014 are only generated while the wallet is unlocked. As difficulty below, solo miners typically difficulty bitcoind to get new transactions from the network. There's a lot of bitcoin in Bitcoin that at 2014 glance doesn't look useful, and then only later do you realize why it's so important.
This the the only known reduction in the total mined supply of Bitcoin. How does Bitcoin make money? There are many other alternative cryptocurrencies to choose from. Not too sure about the DragonMint machine lots of negative press out there but Slush does sound reputable. This result can change depending on your own electricity cost, the change in mining difficulty and most importantly the change in the price of Bitcoin. Thanks for sharing your analysis. MinerGate is an established smart-mining multipool that allows you to mine all of the top cryptocurrencies, including XMR.
download bitcoin aliens app В»
This leaves receivers only two bitcoin ways to issue refunds:. One 2014 strategy is to connect to a number of full nodesand send the requests to each node. 2014 rules allow null bitcoin outputs up to the maximum allowed pubkey script size of 10, bytes provided they follow all other consensus rulessuch as not having any data pushes larger than bytes. It could be argued that a large reason why mining was profitable came simply from BTC gaining value. An interesting source of double-spend risk analysis can be acquired by connecting to large numbers of Bitcoin peers to track how transactions and difficulty differ from each other. Estimate we are all fighting estimate other and we are all difficulty bastards What would be the necessary amount of energy required to run this system?
These modifications change the hash of the transaction. Since the hash is part of the block, a transaction has a fixed hash and cannot be modified by malleability once it has been mined into a block. Unless the whole block is orphaned, of course. This is about the total power consumption of Cambodia. Other estimates are at Hacker News. If they successfully mine a block, why not submit it themselves so they can claim the full mining reward, rather than splitting it? The main reason is the coinbase transaction has the pool's address, not the miner's address.
If the miner submits the block bypassing the pool, the reward still goes to the pool. And if the miner changes the address, the hash is no longer valid. For instance, a pool can pay out the exact amount earned from a block or an average amount. Or a pool can pay a fixed amount per share. A pool can weight shares by time to avoid miners switching between pools mid-block. These different systems can balance risk between the miners and the pool operator and adjust the variance of payments.
For details, see the Bitcoin wiki here or here. First it contains the block height 0x or , which is required for version 2. This is followed by a timestamp. Next is 8 bytes of the two nonces. This is followed by apparently-random data and then the text "Happy NY!
Originally, the output scripts were all pay-to-pubkey , with the script: This script puts the public key itself in the script. This script only includes the public key hash the address and requires the redeemer to provide the public key. To see the difference, compare the output scripts in this transaction and this transaction. The merkle tree is a critical optimization for Bitcoin - it's what makes SPV wallets like Multibit possible.
In fact the among the experts there's consensus that the merkle tree should have extended into transactions themselves, so that all the inputs and outputs of a transaction would be committed to via a merkle tree.
In the future this will probably be done, and is needed for things like fraud proofs. Incidentally, here's a fairly complete and "pythonistic" Python library for Bitcoin: Network code is still in flux, but there exists an RPC module for use with a local bitcoin node. A simple example of that type of use is in my dust-b-gone. As for the overhead of using a merkle tree to hash some data verses hashing it in one go, it's roughly speaking double the work. It's easy to see why if you remember your sum-of-series stuff from highschool: Thanks for the detailed comments, Peter.
The funny thing is I realized when looking at mining pool computations that the Merkle hash was in fact useful. I thought I had removed the part about it being pointless, but I guess not: There's a lot of stuff in Bitcoin that at first glance doesn't look useful, and then only later do you realize why it's so important.
On the other hand, there's also a lot of stuff that makes you wonder WTF was Satoshi smoking I'm still quite new to the mechanics of Bitcoins and pools, but hypothetically, could a malicious pool participant be programmed to send hashes that met the pool difficulty to the pool, but keep the Bitcoin difficulty-level hashes for himself and submit them directly to the Bitcoin network?
Are there any checks that could be implemented on the pool software to make sure malicious clients aren't stealing successes? Thanks for this article, especially footnote number 7.
In reading about this mining pool stuff, I had always wondered why people didn't just cheat and not submit the winning share to their pool if they happened to find it. Thanks to your article I finally duh figured it out. Ken, this is great stuff, I find myself already looking forward to your next post, and cutting and pasting the code to play with it.
I was under the impression that the odd one out was hashed with itself and that hash was then included as a branch. As soon as I posted I saw the line of code that does it. So yes you do hash the odd transaction at the end to itself. Thanks again for a great post, with code!!
I'm a little confused if the pool informs the clients of all of the transactions it wants to include in a block if one is found. From the wiki https: Is that the hash that is used to sign each input? I'm trying to get my head around inputs that are signed by different owners and at different times. Is the above double hash used to sign each input no matter who the owner , or are different hashes signed?
I rewrote your sample Python hashing script for merkle trees in PHP if anyone is interested. Hello Ken, excellent deep article. I wonder how you get the figure of 11 million years on average? Very detailed article for those wanting to know how mining actually works. Will point readers here when they ask how mining results in BTC. Ken, how is it your articles on Bitcoin are always the only ones that provide answers to the questions I have in my head?
The [2] need more precision. Helped me understand the details. I trained a machine learning classifier to answer the question "Is the nonce greater than k", similar but not the same as your Notes and references point 1. Thanks again for a great post. The statement that the Merkle tree idea is patented is rather pointless, because the patent expired more than 10 years ago it was issued in and patents last for 20 years.
Ken, can you please explain more about "Creating a block for a pool", coinb1 and extranonce1 is ok, but from where you get - "e4" and continue "cfa I try to write it on VB.
Good night, excuse me my poor english, My name is Gilberto , I'm Mexican and I'm Master student, and I'm trying to investigate a new form to do mining of Bitcoins, but I have a lot of questions about How is the manual procedure to do mining of Bitcoins? I want to do a embedded system in a 7. E64G Ephifany core card and use computing paralell and I not have idea How begin?
Gilberto, first lean how to hash SHA1 manually from this guy and then go from there. Miners normally all start with the same nonce value and then count through as fast as they can. But other parts of the block will be different, so they're trying different blocks. In a mining pool, miners probably get the same transactions but a different extranonce1, to avoid different miners duplicating work.
But different miners could get different transactions, if the pool operator updates the block as time goes on. Two blocks mined at the same time could have the same number of transactions, or could be totally different. It's possible they have no transactions in common. Or they could have identical transactions. Yes, a single miner can pick the transactions that go into the block the miner is working on.
Normally the miner would pick a bunch of transactions to get more fees but they don't need to. Yes, it's like a lotto. Someone with a slow PC could get lucky and mine a block or even someone mining on an old punchcard computer.
It's just very unlikely since fast hardware gives you many more chances to "win". Great article and given that you're still answering questions 3 years later I thought I'd repeat an unanswered question from earlier that piqued my curiosity. That's a good question, but no. The coinbase transaction contains the pool owner's scriptPubKey, so the pool owner is the only one who can access the reward. If the miner changes the scriptPubKey, the hash is no longer valid.
There's a theoretical attack where the miner throws away a fully-successful hash so nobody collects. Then the miner gets paid for the partially successful hashes but the pool owner doesn't get the reward payout. This is known as the withholding attack.
Since it doesn't benefit the miner, it's not too useful as an attack. I believe the mining pool gives each miner different extranonce1 values. A miner can then run through all the nonce and extranonce2 values without duplicating work. Because of the different values for extranonce1, each miner can work with the same transactions but will still be generating unique blocks.
This comment is extract from another blog: There is no precise nonce finding protocol. The miner can arbitrarily choose a nonce c to perform the hashing operation. Mining is a mathematical game where the goal is to make the result of the hash function smaller than a given number this is what "a result starting with x zeros" is looking for.
The number is directly based on the current Bitcoin network difficulty and changes every two weeks to keep average block finding time at seconds. Most nonce generators just increment by 1 but the key is where they start. If you are solo mining, you can pick a random number. If you are mining with multiple devices or you are a pool administrator, you have to divide the work to avoid calculating the same hash twice make sure they never use the same c.
I'm a big fan of your articles. Can I translate some of them to my blog in Portuguese? Thank you in advance. If you send me a link to your translation, I can add it to this page. The text above the figure should mention this. Jonathan, you're right - there's an extra digit for the lock time in the diagram.
You get points for studying the diagram more carefully than anyone else ;- Your post got through fine, by the way along with a lot of spam I'm constantly removing. Hi first of all thanks for this really fantastic series,this really helpful. You pick which valid transactions you want to put in the block. You pick a roughly accurate value for the timestamp. The Merkle root is formed by hashing pairs of transactions and then hashing pairs of hashes until you have a single value see footnote 4 for details.
Then you try to hash the resulting block with different nonces, hoping to find a successful block If you succeed in mining, you send the block to the Bitcoin network. Since the network is peer-to-peer, you send your successfully mined block to other computers peers in the Bitcoin network, who send it to other computers, until everyone has received it in a few seconds.
Peers are always sharing blocks, which is how they get passed around the network, and there is nothing special about you sharing a block that you just mined versus a block that you received from someone else. Miners will then start using your block as the previous block for their mining, which is how your mined block becomes part of the blockchain.
Hi thanks for reply your explanation was helpful but there is 2 point that steel unclear for me. No, because the address to grant the reward is the pools address. Bitfury BFC55 comes in different configurations, model assumes a 0. Bitfury 28nm comes in different configurations, model assumes a 0. Bitfury BFC16 comes in different configurations, model assumes a 0. KnCMiner Solar comes in different configurations, model assumes a 0.
The model presented in this post makes one assumption: Hypothetically, if a machine is first put online, and if it is immediately decommissioned within the same phase eg. The worst line never intersects the threshold. The least efficient machines remain profitable during their entire phase of production. We can calculate the upper bound for the global electricity consumption of Bitcoin miners by assuming they deploy the least efficient hardware of their time and never upgrade it.
As to the lower bound it can be calculated by assuming everyone has upgraded to the most efficient hardware. This may sound like a lot of electricity but when considering the big picture I believe Bitcoin mining is not wasteful. Lastly, when modeling the costs and revenues of a miner over its entire life such as the Antminer S5, we find out that the hardware cost is as high as, if not higher than its lifetime electricity cost.
On 11 March I removed the assumption that sales of A dwindled down to practically zero post-June , because although sales volume did decrease I do not have precise metrics to justify it.
On 30 March I added the comparison to the electricity consumption of decorative Christmas lights. On 4 June I added all miners released in the last 2. The chart covers the period 15 December to 26 February Starting as early as December is sufficient for accurate modeling because only one ASIC released in phase 0 is still profitable: All others are no longer profitable.
The daily hash rate data was obtained from Quandl ; the curve was smoothed out by calculating each day as the average of this day and the 9 previous ones. It is logical to assume miners seek geographical locations with the cheapest electricity. Mining hardware manufacturers only sell one generation of miners at any given time. Usually it is a result of producing and selling small batches one by one, as Bitmain and Canaan have done.
But it is also a result of aggressive competition: The profitability threshold in joule per gigahash is calculated as such: Neptune, RockerBox, and A It could be argued that a large reason why mining was profitable came simply from BTC gaining value. I reached out to Spondoolies CEO Guy Corem to get official confirmation of when their sales stopped, but have not received a reply so far. I like it - I've not run the estimates on mining for a while busy with other stuff , but I just found one from about 2 years ago where I'd estimated a best case of MW, and a more likely MW at that point in time.
Do your energy figures allow for just the ASIC characteristic or have you factored other inefficiencies especially in PSUs, cooling, etc. The lower bound, by nature, needs to assume the overhead is zero. Thanks for sharing your analysis. It helped me clear a lot of misunderstandings I had. I reviewed the income-antminer-s5. What is your take on that? Ok, I just found your other article at http: Although my full response to and criticism against Digiconomist is at http: I run the SRSroccoReport.
I see you have had a debate with Digiconomist on the energy consumption and cost to produce bitcoin. I am trying to find out a basic cost of production for bitcoin and ethereum, as I believe this would at least provide a floor for their price. Can you reply here or contact me at SRSroccoReport gmail. I would enjoy hearing what you would gauge as a current total cost to produce bitcoin and ethereum. I do realize their costs will continue to increase as time goes by, but it would be helpful in comparing cost of production to their market price An Antminer S9 operates at 0.
This doesn't count the cost of the hardware which has to be amortized over the lifetime of a miner. But even this number doesn't account for other smaller expenses: Thank you for sharing!
Really enjoyed reading your analysis. I would imagine the global mean is even higher. The problem of estimating Bitcoin energy consumption is a lack of a central register with all active machines. If you're going to derive energy consumption from actual hash you're going to have a pretty big error on the tail. This is the part with most older machines, that relatively have the most impact on total energy use eg. The author heavily relies on economic assumptions in determining the activity of these older machines, which adds a lot of uncertainty regarding this so-called "bound".
IMO this hasn't been properly disclaimed in the article. Still I'm happy with it, since it also validates the need for an economic indicator given the reliance on profitability assumptions. Yes, but it doesn't disclose uncertainty surrounding that number. Average cost per KWh are an estimate, not a given. Only the lower bound is an actual bound. The way it's presented makes it seem like the upper bound is of equal strenght as the lower bound. While the lower bound only has some performance uncertainty surrounding it, but the upper bound is a diffent story.
It's not that solid. On top of the previous the number is also sensitive to timing after all there's no guarantee to when machines are actually deployed - shipping and setting up take time too and hashrate measurement errors.
Yes the upper bound is influenced by the assumed cost of electricity, and there is some uncertainty about the cost. I disclose this assumption in multiple places. But I do not believe a lower cost would have a significant impact on the tail.
Almost all desktop wallets can associate with bitcoin: URIs , so spenders can click a link to pre-fill the payment screen. This also works with many mobile wallets , but it generally does not work with web-based wallets unless the spender installs a browser extension or manually configures a URI handler. Most mobile wallets support scanning bitcoin: URIs encoded in a QR code, and almost all wallets can display them for accepting payment.
While also handy for online orders, QR Codes are especially useful for in-person purchases. Special care must be taken to avoid the theft of incoming payments.
To specify an amount directly for copying and pasting, you must provide the address , the amount, and the denomination. An expiration time for the offer may also be specified. Indicating the denomination is critical. Choosing between each unit is widely supported, but other software also lets its users select denomination amounts from some or all of the following options:.
URI scheme defined in BIP21 eliminates denomination confusion and saves the spender from copying and pasting two separate values. It also lets the payment request provide some additional information to the spender.
Only the address is required, and if it is the only thing specified, wallets will pre-fill a payment request with it and let the spender enter an amount. The amount specified is always in decimal bitcoins BTC. Two other parameters are widely supported. The message parameter is generally used to describe the payment request to the spender.
Both the label and the message must be URI encoded. All four parameters used together, with appropriate URI encoding, can be seen in the line-wrapped example below. The URI scheme can be extended, as will be seen in the payment protocol section below, with both new optional and required parameters.
Programs accepting URIs in any form must ask the user for permission before paying unless the user has explicitly disabled prompting as might be the case for micropayments. QR codes are a popular way to exchange bitcoin: URIs in person, in images, or in videos. Most mobile Bitcoin wallet apps, and some desktop wallets , support scanning QR codes to pre-fill their payment screens. The figure below shows the same bitcoin: The QR code can include the label and message parameters—and any other optional parameters—but they were omitted here to keep the QR code small and easy to scan with unsteady or low-resolution mobile cameras.
The error correction is combined with a checksum to ensure the Bitcoin QR code cannot be successfully decoded with data missing or accidentally altered, so your applications should choose the appropriate level of error correction based on the space you have available to display the code.
Low-level damage correction works well when space is limited, and quartile-level damage correction helps ensure fast scanning when displayed on high-resolution screens. The payment protocol adds many important features to payment requests:. Allows spenders to submit transactions directly to receivers without going through the peer-to-peer network.
This can speed up payment processing and work with planned features such as child-pays-for-parent transaction fees and offline NFC or Bluetooth-based payments. To request payment using the payment protocol , you use an extended but backwards-compatible bitcoin: The r parameter tells payment-protocol-aware wallet programs to ignore the other parameters and fetch a PaymentRequest from the URL provided.
An example CGI program and description of all the parameters which can be used in the Payment Protocol is provided in the Developer Examples Payment Protocol subsection. In this subsection, we will briefly describe in story format how the Payment Protocol is typically used. Charlie, the client, is shopping on a website run by Bob, the businessman. An order total in satoshis , perhaps created by converting prices in fiat to prices in satoshis. A pubkey script to which Charlie should send payment.
URI for Charlie to click to pay. Charlie clicks on the bitcoin: URI in his browser. The unique public key created for the payment request can be used to create a unique identifier.
It then creates a PaymentDetails message with the following information:. The amount of the order in satoshis and the pubkey script to be paid. The time the PaymentDetails message was created plus the time it expires. That PaymentDetails message is put inside a PaymentRequest message. The Payment Protocol has been designed to allow other signing methods in the future.
Among other things, the Payment message contains:. In the case of a dispute, Charlie can generate a cryptographically-proven receipt out of the various signed or otherwise-proven information. The Bitcoin block chain can prove that the pubkey script specified by Bob was paid the specified number of satoshis.
See the Refunds section below for more details. A malicious spender can create one transaction that pays the receiver and a second one that pays the same input back to himself. Only one of these transactions will be added to the block chain , and nobody can say for sure which one it will be.
Two or more transactions spending the same input are commonly referred to as a double spend. Once the transaction is included in a block , double spends are impossible without modifying block chain history to replace the transaction, which is quite difficult. Using this system, the Bitcoin protocol can give each of your transactions an updating confidence score based on the number of blocks which would need to be modified to replace a transaction.
For each block , the transaction gains one confirmation. Since modifying blocks is quite difficult, higher confirmation scores indicate greater protection. The transaction has been broadcast but is still not included in any block.
Zero confirmation transactions unconfirmed transactions should generally not be trusted without risk analysis. Although miners usually confirm the first transaction they receive, fraudsters may be able to manipulate the network into including their version of a transaction.
The transaction is included in the latest block and double-spend risk decreases dramatically. Transactions which pay sufficient transaction fees need 10 minutes on average to receive one confirmation.
However, the most recent block gets replaced fairly often by accident, so a double spend is still a real possibility. The most recent block was chained to the block which includes the transaction. As of March , two block replacements were exceedingly rare, and a two block replacement attack was impractical without expensive mining equipment. The network has spent about an hour working to protect the transaction against double spends and the transaction is buried under six blocks.
Even a reasonably lucky attacker would require a large percentage of the total network hashing power to replace six blocks. Although this number is somewhat arbitrary, software handling high-value transactions, or otherwise at risk for fraud, should wait for at least six confirmations before treating a payment as accepted.
Bitcoin Core provides several RPCs which can provide your program with the confirmation score for transactions in your wallet or arbitrary transactions. For example, the listunspent RPC provides an array of every satoshi you can spend along with its confirmation score.
Although confirmations provide excellent double-spend protection most of the time, there are at least three cases where double-spend risk analysis can be required:.
In the case when the program or its user cannot wait for a confirmation and wants to accept unconfirmed payments. In the case when the program or its user is accepting high value transactions and cannot wait for at least six confirmations or more.
In the case of an implementation bug or prolonged attack against Bitcoin which makes the system less reliable than expected. An interesting source of double-spend risk analysis can be acquired by connecting to large numbers of Bitcoin peers to track how transactions and blocks differ from each other.
Some third-party APIs can provide you with this type of service. For example, unconfirmed transactions can be compared among all connected peers to see if any UTXO is used in multiple unconfirmed transactions , indicating a double-spend attempt, in which case the payment can be refused until it is confirmed. Another example could be to detect a fork when multiple peers report differing block header hashes at the same block height. Your program can go into a safe mode if the fork extends for more than two blocks , indicating a possible problem with the block chain.
For more details, see the Detecting Forks subsection. Another good source of double-spend protection can be human intelligence. For example, fraudsters may act differently from legitimate customers, letting savvy merchants manually flag them as high risk. Your program can provide a safe mode which stops automatic payment acceptance on a global or per-customer basis.
Occasionally receivers using your applications will need to issue refunds. The obvious way to do that, which is very unsafe, is simply to return the satoshis to the pubkey script from which they came. Alice wants to buy a widget from Bob, so Bob gives Alice a price and Bitcoin address.
Alice opens her wallet program and sends some satoshis to that address. Bob discovers Alice paid too many satoshis. Being an honest fellow, Bob refunds the extra satoshis to the mjSk… address. Now the refund is a unintentional donation to the company behind the centralized wallet , unless Alice opens a support ticket and proves those satoshis were meant for her. This leaves receivers only two correct ways to issue refunds:.
If an address was copy-and-pasted or a basic bitcoin: URI was used, contact the spender directly and ask them to provide a refund address. Many receivers worry that their satoshis will be less valuable in the future than they are now, called foreign exchange forex risk. If your application provides this business logic, it will need to choose which outputs to spend first.
There are a few different algorithms which can lead to different results. A merge avoidance algorithm makes it harder for outsiders looking at block chain data to figure out how many satoshis the receiver has earned, spent, and saved. When a receiver receives satoshis in an output , the spender can track in a crude way how the receiver spends those satoshis.
This is called a merge , and the more a receiver merges outputs , the easier it is for an outsider to track how many satoshis the receiver has earned, spent, and saved. Merge avoidance means trying to avoid spending unrelated outputs in the same transaction.
For persons and businesses which want to keep their transaction data secret from other people, it can be an important strategy. A crude merge avoidance strategy is to try to always pay with the smallest output you have which is larger than the amount being requested. For example, if you have four outputs holding, respectively, , , , and satoshis , you would pay a bill for satoshis with the satoshi output. This way, as long as you have outputs larger than your bills, you avoid merging.
More advanced merge avoidance strategies largely depend on enhancements to the payment protocol which will allow payers to avoid merging by intelligently distributing their payments among multiple outputs provided by the receiver.
Since recent outputs are at the greatest risk of being double-spent , spending them before older outputs allows the spender to hold on to older confirmed outputs which are much less likely to be double-spent. If you spend an output from one unconfirmed transaction in a second transaction, the second transaction becomes invalid if transaction malleability changes the first transaction. In either of the above cases, the receiver of the second transaction will see the incoming transaction notification disappear or turn into an error message.
However, after just a few blocks , a point of rapidly diminishing returns is reached. FIFO does have a small advantage when it comes to transaction fees , as older outputs may be eligible for inclusion in the 50, bytes set aside for no-fee-required high-priority transactions by miners running the default Bitcoin Core codebase.
However, with transaction fees being so low, this is not a significant advantage. The only practical use of FIFO is by receivers who spend all or most of their income within a few blocks , and who want to reduce the chance of their payments becoming accidentally invalid.
Automated recurring payments are not possible with decentralized Bitcoin wallets. Even if a wallet supported automatically sending non-reversible payments on a regular schedule, the user would still need to start the program at the appointed time, or leave it running all the time unprotected by encryption.
This means automated recurring Bitcoin payments can only be made from a centralized server which handles satoshis on behalf of its spenders. In practice, receivers who want to set prices in fiat terms must also let the same centralized server choose the appropriate exchange rate. Non-automated rebilling can be managed by the same mechanism used before credit-card recurring payments became common: In the future, extensions to the payment protocol and new wallet features may allow some wallet programs to manage a list of recurring transactions.
The spender will still need to start the program on a regular basis and authorize payment—but it should be easier and more secure for the spender than clicking an emailed invoice, increasing the chance receivers get paid on time. Currently there are two primary methods of validating the block chain as a client: Full nodes and SPV clients.
Other methods, such as server-trusting methods, are not discussed as they are not recommended. This security model assures the validity of the block chain by downloading and validating blocks from the genesis block all the way to the most recently discovered block.
Due to the computational difficulty required to generate a new block at the tip of the chain, the ability to fool a full node becomes very expensive after 6 confirmations. An alternative approach detailed in the original Bitcoin paper is a client that only downloads the headers of blocks during the initial syncing process and then requests transactions from full nodes as needed.
This scales linearly with the height of the block chain at only 80 bytes per block header , or up to 4. As described in the white paper, the merkle root in the block header along with a merkle branch can prove to the SPV client that the transaction in question is embedded in a block in the block chain.
This does not guarantee validity of the transactions that are embedded. Instead it demonstrates the amount of work required to perform a double-spend attack. The SPV client knows the merkle root and associated transaction information, and requests the respective merkle branch from a full node. Once the merkle branch has been retrieved, proving the existence of the transaction in the block , the SPV client can then look to block depth as a proxy for transaction validity and security.
The cost of an attack on a user by a malicious node who inserts an invalid transaction grows with the cumulative difficulty built on top of that block , since the malicious node alone will be mining this forged chain.
If implemented naively, an SPV client has a few important weaknesses. First, while the SPV client can not be easily fooled into thinking a transaction is in a block when it is not, the reverse is not true.
A full node can simply lie by omission, leading an SPV client to believe a transaction has not occurred. This can be considered a form of Denial of Service. One mitigation strategy is to connect to a number of full nodes , and send the requests to each node.
However this can be defeated by network partitioning or Sybil attacks, since identities are essentially free, and can be bandwidth intensive. Care must be taken to ensure the client is not cut off from honest nodes. Second, the SPV client only requests transactions from full nodes corresponding to keys it owns.
If the SPV client downloads all blocks and then discards unneeded ones, this can be extremely bandwidth intensive. If they simply ask full nodes for blocks with specific transactions, this allows full nodes a complete view of the public addresses that correspond to the user. This is a large privacy leak, and allows for tactics such as denial of service for clients, users, or addresses that are disfavored by those running full nodes , as well as trivial linking of funds.
A client could simply spam many fake transaction requests, but this creates a large strain on the SPV client , and can end up defeating the purpose of thin clients altogether. To mitigate the latter issue, Bloom filters have been implemented as a method of obfuscation and compression of block data requests.
A Bloom filter is a space-efficient probabilistic data structure that is used to test membership of an element. The data structure achieves great data compression at the expense of a prescribed false positive rate. A Bloom filter starts out as an array of n bits all set to 0. A set of k random hash functions are chosen, each of which output a single integer between the range of 1 and n. When adding an element to the Bloom filter , the element is hashed k times separately, and for each of the k outputs, the corresponding Bloom filter bit at that index is set to 1.
Querying of the Bloom filter is done by using the same hash functions as before. If all k bits accessed in the bloom filter are set to 1, this demonstrates with high probability that the element lies in the set. Clearly, the k indices could have been set to 1 by the addition of a combination of other elements in the domain, but the parameters allow the user to choose the acceptable false positive rate.
Removal of elements can only be done by scrapping the bloom filter and re-creating it from scratch. Rather than viewing the false positive rates as a liability, it is used to create a tunable parameter that represents the desired privacy level and bandwidth trade-off. A SPV client creates their Bloom filter and sends it to a full node using the message filterload , which sets the filter for which transactions are desired.
The command filteradd allows addition of desired data to the filter without needing to send a totally new Bloom filter , and filterclear allows the connection to revert to standard block discovery mechanisms.
If the filter has been loaded, then full nodes will send a modified form of blocks , called a merkle block. The merkle block is simply the block header with the merkle branch associated with the set Bloom filter. An SPV client can not only add transactions as elements to the filter, but also public keys , data from signature scripts and pubkey scripts , and more. This enables P2SH transaction finding. If a user is more privacy-conscious, he can set the Bloom filter to include more false positives, at the expense of extra bandwidth used for transaction discovery.
If a user is on a tight bandwidth budget, he can set the false-positive rate to low, knowing that this will allow full nodes a clear view of what transactions are associated with his client. Used in most Android wallets. Bloom filters were standardized for use via BIP Review the BIP for implementation details. There are future proposals such as Unspent Transaction Output UTXO commitments in the block chain to find a more satisfactory middle-ground for clients between needing a complete copy of the block chain , or trusting that a majority of your connected peers are not lying.
UTXO commitments would enable a very secure client using a finite amount of storage using a data structure that is authenticated in the block chain.
These type of proposals are, however, in very early stages, and will require soft forks in the network. Until these types of operating modes are implemented, modes should be chosen based on the likely threat model, computing and bandwidth constraints, and liability in bitcoin value.
The Bitcoin network protocol allows full nodes peers to collaboratively maintain a peer-to-peer network for block and transaction exchange. Full nodes download and verify every block and transaction prior to relaying them to other nodes. Archival nodes are full nodes which store the entire blockchain and can serve historical blocks to other nodes. Pruned nodes are full nodes which do not store the entire blockchain. Many SPV clients also use the Bitcoin network protocol to connect to full nodes.
Consensus rules do not cover networking, so Bitcoin programs may use alternative networks and protocols, such as the high-speed block relay network used by some miners and the dedicated transaction information servers used by some wallets that provide SPV -level security. To provide practical examples of the Bitcoin peer-to-peer network , this section uses Bitcoin Core as a representative full node and BitcoinJ as a representative SPV client.
Both programs are flexible, so only default behavior is described. The response to the lookup should include one or more DNS A records with the IP addresses of full nodes that may accept new incoming connections. For example, using the Unix dig command:. The DNS seeds are maintained by Bitcoin community members: In either case, nodes are added to the DNS seed if they run on the default Bitcoin ports of for mainnet or for testnet.
For this reason, programs should not rely on DNS seeds exclusively. Once a program has connected to the network , its peers can begin to send it addr address messages with the IP addresses and port numbers of other peers on the network , providing a fully decentralized method of peer discovery. Bitcoin Core keeps a record of known peers in a persistent on-disk database which usually allows it to connect directly to those peers on subsequent startups without having to use DNS seeds.
However, peers often leave the network or change IP addresses, so programs may need to make several different connection attempts at startup before a successful connection is made. This can add a significant delay to the amount of time it takes to connect to the network , forcing a user to wait before sending a transaction or checking the status of payment.
Bitcoin Core also tries to strike a balance between minimizing delays and avoiding unnecessary DNS seed use: Both Bitcoin Core and BitcoinJ also include a hardcoded list of IP addresses and port numbers to several dozen nodes which were active around the time that particular version of the software was first released. Bitcoin Core will start attempting to connect to these nodes if none of the DNS seed servers have responded to a query within 60 seconds, providing an automatic fallback option.
As a manual fallback option, Bitcoin Core also provides several command-line connection options, including the ability to get a list of peers from a specific node by IP address, or to make a persistent connection to a specific node by IP address.
See the -help text for details. BitcoinJ can be programmed to do the same thing. Connecting to a peer is done by sending a version message , which contains your version number, block , and current time to the remote node. The remote node responds with its own version message. Then both nodes send a verack message to the other node to indicate the connection has been established.
Once connected, the client can send to the remote node getaddr and addr messages to gather additional peers. In order to maintain a connection with a peer , nodes by default will send a message to peers before 30 minutes of inactivity.
If 90 minutes pass without a message being received by a peer , the client will assume that connection has closed. Before a full node can validate unconfirmed transactions and recently-mined blocks , it must download and validate all blocks from block 1 the block after the hardcoded genesis block to the current tip of the best block chain.
In this case, a node can use the IBD method to download all the blocks which were produced since the last time it was online. Bitcoin Core uses the IBD method any time the last block on its local best block chain has a block header time more than 24 hours in the past. Bitcoin Core up until version 0. The goal is to download the blocks from the best block chain in sequence.
The first time a node is started, it only has a single block in its local best block chain —the hardcoded genesis block block 0. This node chooses a remote peer , called the sync node , and sends it the getblocks message illustrated below.
In the header hashes field of the getblocks message , this new node sends the header hash of the only block it has, the genesis block 6fe2… in internal byte order. It also sets the stop hash field to all zeroes to request a maximum-size response.
Upon receipt of the getblocks message , the sync node takes the first and only header hash and searches its local best block chain for a block with that header hash. It finds that block 0 matches, so it replies with block inventories the maximum response to a getblocks message starting from block 1. It sends these inventories in the inv message illustrated below. Inventories are unique identifiers for information on the network. Each inventory contains a type field and the unique identifier for an instance of the object.
The block inventories appear in the inv message in the same order they appear in the block chain , so this first inv message contains inventories for blocks 1 through For example, the hash of block 1 is … as seen in the illustration above. The IBD node uses the received inventories to request blocks from the sync node in the getdata message illustrated below. Upon receipt of the getdata message , the sync node replies with each of the blocks requested.
Each block is put into serialized block format and sent in a separate block message. The first block message sent for block 1 is illustrated below. When it has requested every block for which it has an inventory , it sends another getblocks message to the sync node requesting the inventories of up to more blocks. This second getblocks message contains multiple header hashes as illustrated below:. Upon receipt of the second getblocks message , the sync node searches its local best block chain for a block that matches one of the header hashes in the message, trying each hash in the order they were received.
If it finds a matching hash, it replies with block inventories starting with the next block from that point. But if there is no matching hash besides the stopping hash , it assumes the only block the two nodes have in common is block 0 and so it sends an inv starting with block 1 the same inv message seen several illustrations above. This fork detection becomes increasingly useful the closer the IBD node gets to the tip of the block chain.
When the IBD node receives the second inv message , it will request those blocks using getdata messages. The sync node will respond with block messages. Then the IBD node will request more inventories with another getblocks message —and the cycle will repeat until the IBD node is synced to the tip of the block chain. At that point, the node will accept blocks sent through the regular block broadcasting described in a later subsection.
The primary advantage of blocks-first IBD is its simplicity. The primary disadvantage is that the IBD node relies on a single sync node for all of its downloading. This has several implications:. All requests are made to the sync node , so if the sync node has limited upload bandwidth, the IBD node will have slow download speeds. The sync node can send a non-best but otherwise valid block chain to the IBD node.
Bitcoin Core ships with several block chain checkpoints at various block heights selected by developers to help an IBD node detect that it is being fed an alternative block chain history—allowing the IBD node to restart its download earlier in the process.
Closely related to the download restarts, if the sync node sends a non-best but otherwise valid block chain , the chain will be stored on disk, wasting space and possibly filling up the disk drive with useless data.
Orphan blocks are stored in memory while they await validation, which may lead to high memory use. All of these problems are addressed in part or in full by the headers-first IBD method used in Bitcoin Core 0.
The table below summarizes the messages mentioned throughout this subsection. The links in the message field will take you to the reference page for that message. The goal is to download the headers for the best header chain , partially validate them as best as possible, and then download the corresponding blocks in parallel. This solves several problems with the older blocks-first IBD method.
In the header hashes field of the getheaders message , the new node sends the header hash of the only block it has, the genesis block 6fe2… in internal byte order. Upon receipt of the getheaders message , the sync node takes the first and only header hash and searches its local best block chain for a block with that header hash.
It finds that block 0 matches, so it replies with 2, header the maximum response starting from block 1. It sends these header hashes in the headers message illustrated below. The IBD node can partially validate these block headers by ensuring that all fields follow consensus rules and that the hash of the header is below the target threshold according to the nBits field. Full validation still requires all transactions from the corresponding block.
After the IBD node has partially validated the block headers , it can do two things in parallel:. Those headers can be immediately validated and another batch requested repeatedly until a headers message is received from the sync node with fewer than 2, headers , indicating that it has no more headers to offer.
As of this writing, headers sync can be completed in fewer than round trips, or about 32 MB of downloaded data. Once the IBD node receives a headers message with fewer than 2, headers from the sync node , it sends a getheaders message to each of its outbound peers to get their view of best header chain. By comparing the responses, it can easily determine if the headers it has downloaded belong to the best header chain reported by any of its outbound peers.
While the IBD node continues downloading headers , and after the headers finish downloading, the IBD node will request and download each block. The IBD node can use the block header hashes it computed from the header chain to create getdata messages that request the blocks it needs by their inventory. Although not all full nodes may store all blocks. This allows it to fetch blocks in parallel and avoid having its download speed constrained to the upload speed of a single sync node.
To spread the load between multiple peers , Bitcoin Core will only request up to 16 blocks at a time from a single peer. Combined with its maximum of 8 outbound connections, this means headers-first Bitcoin Core will request a maximum of blocks simultaneously during IBD the same maximum number that blocks-first Bitcoin Core requested from its sync node.
Once the IBD node is synced to the tip of the block chain , it will accept blocks sent through the regular block broadcasting described in a later subsection. When a miner discovers a new block , it broadcasts the new block to its peers using one of the following methods:.