| Current File : //usr/lib/python3/dist-packages/s3transfer/__pycache__/bandwidth.cpython-312.pyc |
�
�<�e= � � � d dl Z d dlZ G d� de� Z G d� d� Z G d� d� Z G d� d � Z G d
� d � Z G d � d
� Z G d� d� Z G d� d� Z
y)� Nc � � � e Zd Z� fd�Z� xZS )�RequestExceededExceptionc �d �� || _ || _ dj ||� }t �| � |� y)a� Error when requested amount exceeds what is allowed
The request that raised this error should be retried after waiting
the time specified by ``retry_time``.
:type requested_amt: int
:param requested_amt: The originally requested byte amount
:type retry_time: float
:param retry_time: The length in time to wait to retry for the
requested amount
z<Request amount {} exceeded the amount available. Retry in {}N)�
requested_amt�
retry_time�format�super�__init__)�selfr r �msg� __class__s ��6/usr/lib/python3/dist-packages/s3transfer/bandwidth.pyr
z!RequestExceededException.__init__ s8 �� � +���$���L�S�S� �:�
�� ���� � )�__name__�
__module__� __qualname__r
�
__classcell__)r
s @r r r s
�� � � r r c � � e Zd ZdZy)� RequestTokenzDA token to pass as an identifier when consuming from the LeakyBucketN)r r r �__doc__� r r r r ' s � �N�r r c � � e Zd Zd� Zd� Zy)� TimeUtilsc �* � t j � S )zgGet the current time back
:rtype: float
:returns: The current time in seconds
)�time�r s r r zTimeUtils.time. s � � �y�y�{�r c �, � t j |� S )zwSleep for a designated time
:type value: float
:param value: The time to sleep for in seconds
)r �sleep)r �values r r zTimeUtils.sleep6 s � � �z�z�%� � r N)r r r r r r r r r r - s � ��!r r c � � e Zd Zdd�Z dd�Zy)�BandwidthLimiterNc �D � || _ || _ |�t � | _ yy)a Limits bandwidth for shared S3 transfers
:type leaky_bucket: LeakyBucket
:param leaky_bucket: The leaky bucket to use limit bandwidth
:type time_utils: TimeUtils
:param time_utils: Time utility to use for interacting with time.
N)�
_leaky_bucket� _time_utilsr )r � leaky_bucket�
time_utilss r r
zBandwidthLimiter.__init__@ s) � � *���%��� � � (�{�D� � r c �n � t || j || j � }|s|j � |S )a� Wraps a fileobj in a bandwidth limited stream wrapper
:type fileobj: file-like obj
:param fileobj: The file-like obj to wrap
:type transfer_coordinator: s3transfer.futures.TransferCoordinator
param transfer_coordinator: The coordinator for the general transfer
that the wrapped stream is a part of
:type enabled: boolean
:param enabled: Whether bandwidth limiting should be enabled to start
)�BandwidthLimitedStreamr# r$ �disable_bandwidth_limiting)r �fileobj�transfer_coordinator�enabled�streams r �get_bandwith_limited_streamz,BandwidthLimiter.get_bandwith_limited_streamN s: � � (� �T�'�'�)=�t�?O�?O�
�� � � -� -� /��
r �N)T)r r r r
r. r r r r! r! ? s � � +� 6:�r r! c �\ � e Zd Z dd�Zd� Zd� Zd� Zd� Zd� Zd� Z dd �Z
d
� Z d � Z d � Z
d
� Zy)r( Nc � � || _ || _ || _ || _ |�t � | _ d| _ t
� | _ d| _ || _ y)a[ Limits bandwidth for reads on a wrapped stream
:type fileobj: file-like object
:param fileobj: The file like object to wrap
:type leaky_bucket: LeakyBucket
:param leaky_bucket: The leaky bucket to use to throttle reads on
the stream
:type transfer_coordinator: s3transfer.futures.TransferCoordinator
param transfer_coordinator: The coordinator for the general transfer
that the wrapped stream is a part of
:type time_utils: TimeUtils
:param time_utils: The time utility to use for interacting with time
NTr )
�_fileobjr# �_transfer_coordinatorr$ r �_bandwidth_limiting_enabledr �_request_token� _bytes_seen�_bytes_threshold)r r* r% r+ r& �bytes_thresholds r r
z BandwidthLimitedStream.__init__f sU � �0 ��
� )���%9��"�%��� � � (�{�D� �+/��(� *�n��� ��� /�� r c � � d| _ y)z0Enable bandwidth limiting on reads to the streamTN�r4 r s r �enable_bandwidth_limitingz0BandwidthLimitedStream.enable_bandwidth_limiting� s
� �+/��(r c � � d| _ y)z1Disable bandwidth limiting on reads to the streamFNr: r s r r) z1BandwidthLimitedStream.disable_bandwidth_limiting� s
� �+0��(r c �8 � | j s| j j |� S | xj |z
c_ | j | j k r| j j |� S | j � | j j |� S )zhRead a specified amount
Reads will only be throttled if bandwidth limiting is enabled.
)r4 r2 �readr6 r7 � _consume_through_leaky_bucket)r �amounts r r> zBandwidthLimitedStream.read� s � �
�/�/��=�=�%�%�f�-� -�
���F�"�� � � �d� 3� 3� 3��=�=�%�%�f�-� -� �*�*�,��}�}�!�!�&�)�)r c �r � | j j s9 | j j | j | j
� d| _ y | j j �# t $ r/}| j j |j � Y d }~nd }~ww xY w| j j s���i)Nr )
r3 � exceptionr# �consumer6 r5 r r$ r r )r �es r r? z4BandwidthLimitedStream._consume_through_leaky_bucket� s� � � �,�,�6�6�
5��"�"�*�*��$�$�d�&9�&9�� $%�� �� �,�,�6�6� 6�� ,�
5�� � �&�&�q�|�|�4�4��
5�� �,�,�6�6�s �7A& �& B �/%B�B c �$ � | j � y)z6Signal that data being read is being transferred to S3N)r; r s r �signal_transferringz*BandwidthLimitedStream.signal_transferring� s � � �&�&�(r c �$ � | j � y)z:Signal that data being read is not being transferred to S3N)r) r s r �signal_not_transferringz.BandwidthLimitedStream.signal_not_transferring� s � � �'�'�)r c �< � | j j ||� y r/ )r2 �seek)r �where�whences r rJ zBandwidthLimitedStream.seek� s � � �
�
���5�&�)r c �6 � | j j � S r/ )r2 �tellr s r rN zBandwidthLimitedStream.tell� s � ��}�}�!�!�#�#r c � � | j r | j r| j � | j j � y r/ )r4 r6 r? r2 �closer s r rP z BandwidthLimitedStream.close� s1 � � � +� +��0@�0@�
� .� .� 0� �
�
��� r c � � | S r/ r r s r � __enter__z BandwidthLimitedStream.__enter__� s � �� r c �$ � | j � y r/ )rP )r �args�kwargss r �__exit__z BandwidthLimitedStream.__exit__� s � � �
�
� r )Ni )r )r r r r
r; r) r> r? rF rH rJ rN rP rR rV r r r r( r( e sG � � �"�
!0�F0�1�*�(7�$)�*�*�$� ��r r( c �8 � e Zd Z dd�Zd� Zd� Zd� Zd� Zd� Zy) � LeakyBucketNc �� � t |� | _ || _ |�t � | _ t j
� | _ || _ |�t � | _ || _ |�t � | _ yy)a9 A leaky bucket abstraction to limit bandwidth consumption
:type rate: int
:type rate: The maximum rate to allow. This rate is in terms of
bytes per second.
:type time_utils: TimeUtils
:param time_utils: The time utility to use for interacting with time
:type rate_tracker: BandwidthRateTracker
:param rate_tracker: Tracks bandwidth consumption
:type consumption_scheduler: ConsumptionScheduler
:param consumption_scheduler: Schedules consumption retries when
necessary
N) �float� _max_rater$ r � threading�Lock�_lock�
_rate_tracker�BandwidthRateTracker�_consumption_scheduler�ConsumptionScheduler)r �max_rater& � rate_tracker�consumption_schedulers r r
zLeakyBucket.__init__� sl � �. �x� ���%��� � � (�{�D� � �^�^�%��
� )��� � �!5�!7�D� �&;��#� � (�*>�*@�D� '� )r c �l � | j 5 | j j � }| j j |� r | j |||� cddd� S | j
||� r| j |||� n| j ||� cddd� S ddd� y# 1 sw Y yxY w)ac Consume an a requested amount
:type amt: int
:param amt: The amount of bytes to request to consume
:type request_token: RequestToken
:param request_token: The token associated to the consumption
request that is used to identify the request. So if a
RequestExceededException is raised the token should be used
in subsequent retry consume() request.
:raises RequestExceededException: If the consumption amount would
exceed the maximum allocated bandwidth
:rtype: int
:returns: The amount consumed
N) r^ r$ r ra � is_scheduled�,_release_requested_amt_for_scheduled_request� _projected_to_exceed_max_rate�!_raise_request_exceeded_exception�_release_requested_amt�r �amt�
request_token�time_nows r rC zLeakyBucket.consume� s� � �$ �Z�Z� B��'�'�,�,�.�H��*�*�7�7�
�F��H�H�� ��� B� B� �3�3�C��B��6�6�� ��� �2�2�3��A� B� B�� B� B� B�s �AB*� 7B*�*B3c �X � | j j ||� }|| j kD S r/ )r_ �get_projected_rater[ )r rm ro �projected_rates r ri z)LeakyBucket._projected_to_exceed_max_rate s) � � �+�+�>�>�s�H�M�� ����.�.r c �\ � | j j |� | j ||� S r/ )ra � process_scheduled_consumptionrk rl s r rh z8LeakyBucket._release_requested_amt_for_scheduled_request s0 � �
�#�#�A�A� �
� �*�*�3��9�9r c � � |t | j � z }| j j |||� }t ||�� �)N)r r )rZ r[ ra �schedule_consumptionr )r rm rn ro �allocated_timer s r rj z-LeakyBucket._raise_request_exceeded_exception% sF � � �u�T�^�^� 4�4���0�0�E�E� � ��
�
� '� �*�
�
r c �> � | j j ||� |S r/ )r_ �record_consumption_rate)r rm ro s r rk z"LeakyBucket._release_requested_amt. s � � ���2�2�3��A��
r )NNN) r r r r
rC ri rh rj rk r r r rX rX � s. � � �� "� !A�F B�>/�:�
�r rX c �$ � e Zd Zd� Zd� Zd� Zd� Zy)rb c � � i | _ d| _ y)z*Schedules when to consume a desired amountr N)� _tokens_to_scheduled_consumption� _total_waitr s r r
z ConsumptionScheduler.__init__4 s � �02��-� ��r c � � || j v S )z�Indicates if a consumption request has been scheduled
:type token: RequestToken
:param token: The token associated to the consumption
request that is used to identify the request.
)r| )r �tokens r rg z!ConsumptionScheduler.is_scheduled9 s � � ��=�=�=�=r c �| � | xj |z
c_ | j |d�| j |< | j S )a� Schedules a wait time to be able to consume an amount
:type amt: int
:param amt: The amount of bytes scheduled to be consumed
:type token: RequestToken
:param token: The token associated to the consumption
request that is used to identify the request.
:type time_to_consume: float
:param time_to_consume: The desired time it should take for that
specific request amount to be consumed in regardless of previously
scheduled consumption requests
:rtype: float
:returns: The amount of time to wait for the specific request before
actually consuming the specified amount.
)�
wait_duration�time_to_consume)r} r| )r rm r r� s r rv z)ConsumptionScheduler.schedule_consumptionB sA � �&
���O�+�� !� -� -� .�8
��-�-�e�4� � � � r c �| � | j j |� }t | j |d z
d� | _ y)z�Processes a scheduled consumption request that has completed
:type token: RequestToken
:param token: The token associated to the consumption
request that is used to identify the request.
r� r N)r| �pop�maxr} )r r �scheduled_retrys r rt z2ConsumptionScheduler.process_scheduled_consumption\ s= � � �?�?�C�C�E�J�� � � � ��/@� A� A�1�
��r N)r r r r
rg rv rt r r r rb rb 3 s � � �
>� �4
r rb c �< � e Zd Zdd�Zed� � Zd� Zd� Zd� Zd� Z y) r` c �. � || _ d| _ d| _ y)a� Tracks the rate of bandwidth consumption
:type a: float
:param a: The constant to use in calculating the exponentional moving
average of the bandwidth rate. Specifically it is used in the
following calculation:
current_rate = alpha * new_rate + (1 - alpha) * current_rate
This value of this constant should be between 0 and 1.
N)�_alpha�
_last_time�
_current_rate)r �alphas r r
z BandwidthRateTracker.__init__j s � � �� � ��� !��r c �4 � | j �y| j S )zmThe current transfer rate
:rtype: float
:returns: The current tracked transfer rate
� )r� r� r s r � current_ratez!BandwidthRateTracker.current_ratez s � � �?�?� "���!�!�!r c �@ � | j �y| j ||� S )aZ Get the projected rate using a provided amount and time
:type amt: int
:param amt: The proposed amount to consume
:type time_at_consumption: float
:param time_at_consumption: The proposed time to consume at
:rtype: float
:returns: The consumption rate if that amt and time were consumed
r� )r� �*_calculate_exponential_moving_average_rate�r rm �time_at_consumptions r rq z'BandwidthRateTracker.get_projected_rate� s* � � �?�?� "���>�>� �$�
�
r c �v � | j �|| _ d| _ y| j ||� | _ || _ y)a Record the consumption rate based off amount and time point
:type amt: int
:param amt: The amount that got consumed
:type time_at_consumption: float
:param time_at_consumption: The time at which the amount was consumed
Nr� )r� r� r� r� s r ry z,BandwidthRateTracker.record_consumption_rate� sB � � �?�?� "� 1�D�O�!$�D� � � !� L� L� �$�
��� .��r c �J � || j z
}|dk r t d� S ||z S )Nr �inf)r� rZ )r rm r� �
time_deltas r �_calculate_ratez$BandwidthRateTracker._calculate_rate� s. � �(�4�?�?�:�
� ��?�
��<� ��j�!�!r c �~ � | j ||� }| j |z d| j z
| j z z S )N� )r� r� r� )r rm r� �new_rates r r� z?BandwidthRateTracker._calculate_exponential_moving_average_rate� s> � � �'�'��-@�A���{�{�X�%��T�[�[� �D�<N�<N�(N�N�Nr N)g�������?)
r r r r
�propertyr� rq ry r� r� r r r r` r` i s0 � �"� �"� �"�
�$.�$"�Or r` ) r\ r � Exceptionr r r r! r( rX rb r` r r r �<module>r� sk �� � � �y� �, � � !� !�$#� #�Ln� n�bZ� Z�z3
� 3
�lNO� NOr